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given  the  values  of  one  or  more  independent  variables.  Uncertainty 
is  any  inprecision  in  knowledge  resulting  from  causes  such  as  inaccuracy, 
lack  of  validity,  or  bias  in  the  inputs  to  a decision. 

The  analysis  focuses  on  the  specific  decision  problem  of  air  strike 
delay  in  the  context  of  the  CWROEA  strike  warfare  scenario.  The 
baseline  approach  from  which  study  embarked  is  reported  in  Decision 
Analysis  as  an  Element  in  an  Operational  Decision  Aiding  System 
(I®I  Technical  Report  7S-13  for  (IKiR,  AD-A'0lSl'09y.  While  developed 
in  a specific  framework,  the  conclusions,  metho^logy,  and  models 
are  applicable  to  the  broad  class  of  decision  problems  which  requires 
selection  of  the  best  time  to  initiate  a specified  action.  With  modi- 
fication, the  approach  can  be  applied  to  the  multiple-choice  decision 
problem. 

This  is  a report  of  work  acconplished  on  one  task  in  the  project 
entitled,  OperatioMl  Decision  Aids,  which  was  initiated  in  1974  by 
the  Office  of  Kaval  Research  to  develop  aids  for  Navy  conmand  and 
control  functions  and  make  them  available  for  incorporation  in  the 
design  of  future  systems  such  as  the  Task  Force  Conmand  Center  (TFCC). 
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I . INTRODUCTION 


1.1  BACKGROUND 

m 

The  Operational  Decision  Aids  (ODA)  project  is  spon- 
soring development  of  automated  decision-aiding  methodologies 
for  introduction  into  the  fleet.  Many  approaches  and  applicable 
technologies  are  being  investigated,  resulting  in  the  development 
of  decision  aids  which  can  be  broadly  divided  into  two  classes: 
state  models  and  outcome  calculators.  Analytics  is  addressing 
a subject  which  transcends  this  dichotomy;  namely: 

(1)  How  can  nomograph- derived  display  tech- 
niques be  employed  to  aid  in  presentation 
of  results? 

(2)  How  can  uncertainty  in  inputs  and  modeled 
processes  be  best  reflected  in  presentation 
of  results? 

In  order  to  lead  to  practical  results  subject  to  test 
and  evaluation,  the  effort  has  been  directed  at  a particular 
class  of  decision  problems,  and  within  that  class,  at  a specific 

r A 1 
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problem.  The  results  of  this  effort  should  nonetheless  be  inter- 
preted for  the  broader  range  of  decision  problems  to  which  they 
apply. 

One  possible  classification  of  decision  problems  is: 

(1)  Those  that  require  the  selection  of  the 
"best"  action  to  take  at  a specified  time 
(also  called  multiple-choice  problems). 

(2)  Those  that  require  the  "best"  time  at 
which  to  take  a specified  action. 

Many  other  classifications  are  possible;  however,  it 
is  relevant  to  note  that,  using  this  particular  classification, 
the  first  area  has  received  heavy  attention  both  in  the  litera- 
ture and  the  ODA  project,  while  the  second  has  received  virtu- 
ally none.  Additionally,  many  military  decisions  specifically 
address  the  question  of  when  to  act.  Consequently,  our  efforts 
have  focused  on  this  area. 


Within  this  class  of  decision  problems,  we  have  con- 
centrated on  the  question,  "When  is  the  best  time  to  launch  an 
air  strike?"  This  problem  has  received  some  attention  as  noted 
in  "Decision  Analysis  as  an  Element  in  an  Operational  Decision 
Aiding  System"  (Reference  1).  The  structure  presented  there 
serves  as  the  foundation  of  our  efforts.  For  purposes  of  demon- 
stration and  evaluation,  a specific  situation  in  the  ONRODA 
strike  warfare  scenario,  "ONRODA  Strike  Warfare  Scenarios" 
(Reference  2) , has  been  used.  Figure  1 graphically  depicts 
the  decision  situation  employed  in  our  implementation. 


It  is  important  to  note  that  the  methodology  that 
Analytics  has  developed  is  applicable  to  other  "when- to- act" 
decisions  such  as  when  to  launch  an  amphibious  assault,  when 
to  resupply  a ship,  and  when  to  transit  an  ocean  area.  With 
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SEGMENTED  STRIKE  MISSION 


Figure  1.  Implemented  Decision  Situation 


modification,  it  can  also  be  applied  to  conventional  multiple- 
choice  decision  problems  (choose  the  "best"  action  to  take  at 
a specified  time),  and  is  very  appropriate  to  those  types  of 
problems  in  which  the  alternatives  are  time  extensive  (as  in 
an  air  strike) . 

1.2  FEATURES  OF  THE  DECISION  AID 

A decision  problem  can  be  described  by  a set  of 
alternatives,  criteria,  conditioning  elements,  and  a decision 
algorithm.  These  terms  are  defined  as  follows: 

• Alternatives  --  Available  decision  options 
from  which  a selection  is  to  be  made. 

• Criteria  --  Value  measures  upon  which  a 
selection  is  based. 

• Conditioning  Elements  --  The  inputs  and 
assumptions  about  the  conditions  in  the 
real  world. 

• Decision  Algorithm  --  Procedure  that  defines 
how  to  proceed  from  input  to  selection. 

The  nomograph  decision  aid  allows  for  an  unlimited 
number  of  alternatives,  criteria,  and  conditioning  elements. 
Furthermore,  the  conditioning  elements  can  be  continuous  (e.g., 
range,  altitude,  or  readiness),  discrete  (e.g.,  weather  state, 
sea  state,  or  enemy  intent),  or  any  combination  of  discrete  and 
continuous.  The  value  of  a conditioning  element  can  be  expressed 
with  a statement  of  uncertainty  (e.g.,  circular  error  probable, 
variance,  or  standard  error).  Uncertainty  is  imprecise  knowledge 
of  the  real  world  resulting  from  causes  such  as  bias  in  a sensor, 
inaccuracy,  estimation  error,  or  any  other  phenomenon  that  mani- 
fests itself  in  imprecise  knowledge  of  the  real  world. 
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The  aid  explicitly  treats  decisions  that  result  in  an 
action  that  unfolds  over  a time  period  during  which  the  values 
of  the  conditioning  elements  vary.  Thus,  the  aid  specifically 
deals  with  time-varying  statistics  which  are  derived  from  condi- 
tioning elements  defined  as  a function  of  time. 

Additionally,  the  nomograph  decision  aid  allows  the 
user  to  define  a time-dependent  function  that  is  used  to  discount 
the  expected  utilities  achieved  as  the  result  of  the  occurrence 
of  particular  outcomes  at  particular  times.  This  recognizes  that 
there  may  be  time  constraints  associated  with  meeting  tactical 
objectives  that  must  be  factored  into  the  algorithm  to  obtain  a 
realistic  evaluation  of  the  alternative  courses  of  action. 

Finally,  and  most  critically,  the  aid  provides  suffi- 
cient outputs  and  sensitivity  analysis  tools  to  allow  the  indi- 
vidual decision  maker  to  explore  his  own  risk/reward  preferences 
and  to  confirm  or  refute  the  aid's  "judgment."  Specifically, 
the  outputs  allow  use  of  various  decision  strategies  such  as 
maximum  expected  value,  min-max,  above  threshold,  etc. 

1.3  STRUCTURE  OF  DECISION  AID 

The  nomograph  decision  aid  has  been  developed  in  the 
specific  context  of  tho  ONRODA  strike  warfare  scenario  (Appendix 
A).  It  addresses  a segmented  mission  (see  Figure  1)  consisting 
of  five  phases:  (I)  launch,  (II)  enroute  to  target,  (III)  over 
target,  (IV)  return,  and  (V)  landing.  Each  phase  is  assigned  a 
relative  start  time  and  completion  time;  these  times  are  used 
by  the  algorithms  for  proper  selection  of  time-dependent  condi- 
tioning elements  used  in  the  computations. 

The  conditioning  elements  used  are  both  continuous  and 
discrete;  some  are  applicable  only  in  a single  mission  segment, 
while  others  affect  two  or  more  segments  of  the  mission.  The  con- 
ditioning elements  used  in  the  current  implementation  are 
shown  in  Table  1. 
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TABLE  1.  CONDITIONING  ELEMENTS 


NAME  ACRONYM  SEGMENTS  AFFECTED 

OWN  FORCE  READINESS  OFR  ALL 

(3  Aircraft  Type*  1 

ENEMY  AIR  DEFENSE  EAD  II,  III,  IV 

ENEMY  GROUND  DEFENSE  EGD  III 

WEATHER  AT  TARGET  WAT  III 

WEATHER  AT  CARRIER  WAC  V 


r 


I 


Two  criteria  with  associated  weighting  factors  are 
incorporated  in  the  algorithm:  target  destruction  (aircraft 
and  ground  defenses)  and  own  force  losses  (aircraft  and  crew, 
both  killed  and  damaged). 

As  noted  eaTlier,  Analytics'  work  transcends  the 
distinction  between  state  and  outcome  calculator  approaches 
by  incorporating  two  algorithms,  one  based  upon  each  approach 
and  designated  as  the  state  and  outcome  models,  respectively. 
Both  algorithms  fit  into  the  same  formulation  of  the  nomograph 
decision  aid  (see  Figure  2) , using  the  same  inputs  and  providing 
the  same  outputs. 

Both  algorithms  provide  expected  utility  and  a spread 
of  realizable  values  (due  to  uncertainty  in  the  value  of  condi- 
tioning elements  and  in  variations  in  underlying  processes)  for 
each  possible  launch  time.  Additionally,  the  outcome  calcula- 
tion provides  own  force  losses  in  numbers  of  aircraft  on  a 
mission  segment  basis.  Furthermore,  it  can  provide  target 
destruction  achieved  in  numbers  of  enemy  assets  destroyed. 

Each  algorithm  is  supplemented  by  a sensitivity 
analysis  procedure  which  yields  outputs  as  a function  of  any 
one  of  the  conditioning  elements  varied  independently. 

Particularly  with  regard  to  the  outcome  model,  the 
use  of  an  abstract  measure,  such  as  utility,  supplements 
specific  physical  achievements  by  providing  a multi-attribute 
trade-off  among  achievements  according  to  each  separate  cri- 
terion. Other  common  scales,  such  as  equivalent  units  or 
money,  could  be  used  to  effect  the  presentation  of  a single 
measure  of  merit  to  the  decision  maker. 


nn 
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I 1.4  GRAPHICAL  FEATURES 

The  nomograph  decision  aid  has  been  implemented  to 
take  advantage  of  paired  alphanumeric  and  graphical  displays 
available  in  the  test  facility.  Each  tabular  data  entry  dis- 
play is  complemented  by  a color  display  that  presents  the  data 
graphically.  The  decision  aid  outputs  and  sensitivity  analyses 
are  presented  in  color  nomographs.  Finally,  the  attrition 

i 

, outputs  of  the  outcome  calculator  are  presented  in  a multi- 

quadrant color  nomograph,  useful  for  examination  of  "what  if" 

i 

J questions.  Plates  1 through  15  give  examples  of  the  input  and 

output  graphic  displays. 


Many  of  the  displays  provided  in  the  aid  are  intended 
for  use  by  those  staff  officers  (other  than  the  decision  maker) 
who  provide  inputs  to  the  system.  Other  displays  (those  showing 
results  and  sensitivity  analyses)  are  directed  at  the  decision 
maker.  It  is  envisioned  that  the  decision  maker  would  have 
access  to  the  graphical  displays  (e.g.,  Plate  11)  used  by  his 
staff  to  examine  the  inputs  to  the  model  if  he  so  chooses; 
however,  the  tabular  displays  provided  to  enter  inputs  are 
not  expected  to  be  used  by  the  decision  maker. 

Several  graphic  displays  have  been  designed  to  take 
specific  advantage  of  the  color  display  capabilities  available 
in  the  test  bed,  while  other  displays  (though  in  color)  have 
been  designed  for  equal  interpretability  in  a black  and  white 
presentation.  In  particular,  color  has  been  used  primarily 
as  a discriminant,  allowing  a significant  amount  of  information 
to  be  shown  in  one  display  where  a monochromatic  presentation 
would  require  several  displays  (or  one  display  composed  of  sev- 
eral sub-displays).  Alternate  black  and  white  formulations  can 
readily  be  achieved,  opening  the  door  to  a cost  benefit  analy- 


1 . 5 SUMMARY 

The  aid  has  been  developed  and  implemented  to  incor- 
porate significant  capabilities  in  dealing  with  uncertainty  in 
the  decision  process  and  to  apply  state-of-the-art  graphiipal 
display  techniques.  No  limitations  are  placed  on  the  nun)ber 

of  alternatives,  criteria,  and  conditioning  elements  that  can 

/ 

be  accommodated.  Further,  any  restriction  on  the  form  of  the 

I 

conditioning  elements  has  been  removed  by  incorporating/  the 
ability  to  deal  with  both  continuous  and  discrete  case?  --  the 
discrete  conditioning  elements  having  any  arbitrary  number  of 
states.  / 


/ 

The  developed  methodology  has  been  shown  yo  be  com- 
patible with  both  state  and  outcome  calculator  algo^rithms.  At 
the  same  time,  our  work  has  shown  that  the  outcome/ calculator 
approach  has  some  significant  advantages:  f 

f 

(1)  It  is  able  to  output  physically  n^an- 

ingful  results.  / 

(2)  It  is  more  easily  adapted  to  specific 
situations  and  generalized  to  o,iher 
decision  problems. 

/ 

(3)  It  allows  the  decision  problem  to  be 
treated  without  certain  oversimplifi- 
cations required  to  make  the  /state 
model  tractable. 


However,  it  is  also  recognized  that  for  those  deci- 
sions where  development  of  an  outcome  calculator  is  beyond  our 
present  knowledge,  the  state  model  can  provide  a logical  and 
organized  approach  to  attacking  the  problem,  where  there  would 
otherwise  be  a void. 


As  implemented,  the  system  provides  the  opportunity 
for  a wide  variety  of  tests  directed  at  presentation  alternatives. 
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PLATE  1. 


UTILITY  VS.  TIME  — STATE  MODEL 


PLATE  2. 


UTILITY  VS.  TIME  — OUTCOME  MODEL 


PLATE  3. 


THE  LOSS  NOMOGRAPH 


PLATE  4. 


AIRCRAFT  LOSSES  BY  SEGMENT 
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PLATE  5. 


PLATE  6. 


I 


A-«  SENSITIVITY  ANALYSIS 
PLATE  7. 


A-7  SENSITIVITY  ANALYSIS 
PLATE  8. 


PLATE  9. 


PLATE  10. 


r 


WAT  SENSITIVITY  ANALYSIS 


PLATE  11. 


OFR  INPUT  DISPLAY 
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WAC  SENSITIVITY  ANALYSIS 


PLATE  12. 


EAO  INPUT  DISPLAY 


It 

classes  of  algorithms,  and  display  methodologies.  The  ground 
■ has  been  broken  to  examine  the  applicability  of  the  aid  to 

practical  Naval  decision  situations  and,  as  a consequence,  its 
cost-effectiveness  as  a multi-purpose  tool  to  the  Navy  of  the 
future. 


1 
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II.  THEORETIC  FOUNDATION  AND  PRINCIPLES  OF  APPLICATION 


2.1  INTRODUCTION 

Before  applying  nomograph  techniques  to  and  incor- 
porating uncertainty  features  in  decision  aiding  systems,  the 
general  structure  of  decision  models  was  studied  in  order  to 
ensure  that  the  aid  that  would  ultimately  be  designed  would 
not  be  unnecessarily  limited  in  application. 

The  properties  of  the  decision  aid  described  in  the 
DDI  Phase  III  report  (Reference  1)  and  the  literature  on  util- 
ity-based decision  models  was  examined.  From  this  examination, 
certain  specific  properties  required  to  treat  generalized  deci- 
sion problems  were  derived  and  incorporated  into  a generalized 
utility-based  decision  model.  The  model  was  then  restricted  to 
the  specific  implementation  reported  here.  An  outcome  calculator 
analog  was  developed  to  test  transportability  of  our  concepts. 
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2.2 


DEVELOPING  A GENERALIZED  STATE-BASED  DECISION  MODEL 


2.2.1  General  Formulation  of  Current  Models 

Alternatives  (as  defined  in  Section  1)  are  denoted 
as  A^ , where  i varies  from  1 to  d.  An  A^  is  selected  by  maxi- 
mizing the  expected  payoff,  or  utility,  associated  with  it 
according  to  specified  criteria  for  evaluation.  Criteria  are 
denoted  as  Cj , where  j varies  from  1 to  c.  In  the  evaluation, 
criteria  potentially  have  different  importance,  or  weight  (W) . 
Thus,  associated  with  each  Cj  is  a , where 

1 ^ Wj  ^ 0 for  j = 1 , . . . , c 

and 

1 

The  selection  of  A.  also  depends  upon  the  state  of 
the  world  (observed,  hypothesized,  guessed,  etc.).  The  state 
of  the  world  relevant  to  the  decision  at  hand  is  defined  in 
terms  of  conditioning  elements,  which  are  denoted  by  CEj^,  where 
k varies  from  1 to  ce.  Each  CEj^  is  divided  into  a discrete  set 
of  mutually  exclusive  and  exhaustive  states.  The  states  of  a 
particular  CEj^  are  denoted  CEj^  where  H may  vary  from  1 to  s. 

To  each  element  of  a matrix  whose  rows  are  specified 
by  and  whose  columns  are  specified  by  Cj  and  CEj^  a utility 
^ijkA  assigned  (Figure  3). 


STATES 


VALUE  OF 
CONDITIONING 
ELEMENTS 
THAT  DEFINE 
EACH  STATE 


ALTERNATIVES 


S2 


S 


ss 


CRITERIA 

S 

WEIGHT 

Wi 

1 

I'igurc  3.  The  Value  Table 
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This  matrix  of  alternatives  and  utilities  is  a value 
table  (VT).  The  most  general  form  of  a VT  is  one  in  which  each 
column  is  defined  by  every  CEj^  and  the  U's  are  assigned  with  im- 
plicit combination  of  criteria  and  weights;  this  generalization 
obviates  the  1 imitation  of  allowing  only  a weighted  sum  across  cri- 
teria. DDI  did  not  use  this  approach,  nor  will  Analytics;  the 
user  burden  and  reduced  flexibility  in  altering  weights  out- 
weighs the  nicety  of  generality.  Within  this  constraint,  the 
most  general  form  of  a VT  is  one  in  which  one  column  is  defined 
by  every  CEj^  for  each  C j . 

Figure  3 summarizes  the  family  of  VTs  in  our  approach, 
one  for  each  , where  the  tCEj^  map  into  states  of  the  world, 
Sjjj,  with  m varying  from  1 to  ss.  Note  that  the  must  be 
exhaustive  and  mutually  exclusive  with  respect  to  the  (CEj^ 

This  does  not  imply  the  value  of  ss  in  any  way;  however,  for 
the  decision  to  be  non-trivial,  ss  ^ 2. 

To  select  among  the  , we  must  compute  the  expected 
utility  of  each  A^ , namely 

EU(Aj)  i EUj  . D P 
m 


where 


U 


im 


Ew 


j ”ijm 


(remembering  m defines  {krt^}),  and  select  the  maximum.  To  do 
this,  we  must  know  P(Sjjj). 


The  P(Sjjj)  may  be  obtained  from  the  model  user,  or 
they  may  be  generated  by  a simulation  or  analytic  evaluation 
of  the  real  world. 
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DDI  introduces  the  likelihood  table  (LT)  to  enable 

computation  of  the  P(Sjj|).  One  LT  exists  for  each  CEj^.  Further, 

for  each  state  CEj^  ^ of  that  CEj^  a prior  probability  is  stated, 

which  is  labeled  p^^fCE,  5)  such  that 

p r ic  j ^ 


E 


Ppr(“k,t) 


1 


Next,  we  construct  a set  of  indicators,  I , where  n 

n 

varies  from  1 to  N , and  associate  with  each  I likelihood  ratios 

n 

that  would  be  observed,  given  that  CEj^  obtained.  These 
ratios  are  used  to  calculate  posterior  probabilities  of  the 
CE.  . which  are  labeled  p„„(CE.  -),  also  such  that 

K , X,  pO  K , X, 


Indicators  are  required  to  be  independent  so  that 
successive  Bayesian  updates  can  be  computed  validly. 


Upon  receipt  of  any  1^^,  a j^)}  results  which 

combined  with  all  other  sets  for  remaining  CEj^  produces  a pCS^) 
for  all  states.  Obviously,  if  the  VTs  are  segmented  for  each 
CEj^,  as  DDI  has  done,  then  partial  evaluation  of  EU^  can  be 
made  without  completely  determining  which  exists.  However, 
all  such'  partial  evaluations  must  be  made  before  an  can  be 
selected. 


Clearly,  P(Sjj|)  can  be  computed  as  the  joint  prob- 
ability of  all  the  CEj^  that  define  by  multiplying  the 
individual  p(CE|^  as  long  as  the  CEy^  are  independent.  We 
chose  to  restrict  the  formulation  to  such  CEj.  even  though  we 
cam  accommod|^te  dependent^  in  ei  thH  of  two  ways: 


u 

Q 

B 

I 
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• Knowledge  of  the  covariance. 


I 


I 


I 

I 

I 

II 
li 

n 
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• Construction  of  a composite  LT  or  input 
of  a composite  p(CEj^  CE^.,  . 

2. 2. 1.1  Making  Utility  Assignments 

For  any  expected  utility  model  for  decision  making  to 
be  valid,  utility  must  be  defined  on  an  interval  scale.  That 
such  a utility  assignment  is  possible  at  all  is  ensured  only 
if  some  fairly  restrictive  conditions  apply  to  the  preference 
ordering  of  the  outcomes  of  the  action-state  combinations. 

Such  combinations  were  first  described  by  von  Neumann  and  Mor- 
genstern  (Reference  3).  Basically,  they  require  that  there  be 
not  only  a fixed  preference  ordering  of  the  outcomes  but  also 
a reasonably  continuous  ordering  of  all  probability  mixtures 
of  the  outcomes.  (A  probability  mixture  of  outcomes  A and  B 
refers  to  the  situation  where  A obtains  with  some  probability 
p and  B obtains  with  probability  1 - p.)  People  are  never 
totally  consistent  with  the  von  Neumann  and  Morgenstern  axioms 
because  they  invariably  exhibit  vacillation  in  preferences 
between  alternatives  for  which  they  are  nearly  indifferent. 

This  difficulty  can  be  overcome  by  using  a statistical  cri- 
terion for  preference.  Still,  the  axioms  tell  us  only  that 
a utility  assignment  is  possible,  not  how  to  make  it. 

An  acceptable  utility  function  must  generate  utili- 
ties whose  numerical  ordering  mirrors  the  preference  ordering 
of  the  outcomes.  Experimental  studies  that  have  attempted  to 
derive  utility  functions  from  observations  of  preferences  have 
mainly  been  concerned  with  the  monetary  value  of  gambles  (e.g., 
Hosteller  and  Nogee  (Reference  4)  and  Tversky  (Reference  5)),  thus 
simplifying  the  comparison  task  of  the  subject.  One  interesting 
exercise  in  the  estimation  of  utility  was  reported  by  Davidson, 
Suppes,  Siegel  (Reference  6),  using  a linear  programming  model  to 


► 


i 


d rive  utilities  from  stated  preferences  between  musical  selec- 
tions. Although  it  is  interesting  to  try  to  apply  the  linear 
programming  technique  to  determine  utilities,  the  utility 
assignments  ultimately  tend  to  rely  on  authoritative  subjec- 
tive estimates.  Therefore,  it  is  imperative  to  demonstrate 
that  the  utilities  are  consistent  with  preferences  between 
outcomes  and,  perhaps,  even  between  probability  mixtures  of 
outcomes.  Wherever  inconsistencies  are  observed,  it  might 
be  possible  to  revise  the  utilities  by  applying  some  pre- 
defined mathematical  operator. 


Utility  assignments  are  generally  made  using  a scale 
spanning  100  units.  Often  in  comparing  relative  values  among 
decision  options,  given  a state  of  the  world  (a  column  in  the 
table),  the  "best"  value  is  assigned  a value  of  0 and  all 
others  are  assigned  negative  values  up  to  -100.  Thus,  each 
column  always  has  a 0,  and  relative  values  within  rows  have 
no  signif icanf'e.  Such  value  assignments  are  termed  "regrets" 
and  are  used  because  they  are  easier  to  assign  than  utilities. 
Each  entry  represents  how  much  the  decision  maker  would  regret 
the  consequence  of  that  decision  relative  to  the  best  decision 
under  the  circumstances  depicted  by  that  column.  Regrets  are 
derived  from  true  utilities  and  are  relative  rather  than  absolute 
numbers.  Regrets  have  significance  only  within  a column,  e.g., 
within  a single  state  of  the  world. 


One  can  derive  regrets  from  utilities,  but  not  vice 
versa.  The  ranking  of  decisions  using  regrets  is  the  same  as 
if  utilities  were  used,  providing  that  the  same  probabilities 
of  states  are  used  to  evaluate  each  alternative.  This  is 
usually  the  case,  but  in  time-varying  problems,  it  may  not  be. 
Therefore,  this  restriction  must  be  recognized.  If  utilities 
axe  used,  there  is  no  problem  in  applying  time-varying  sta- 
tistics. 
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2. 2. 1.2  Combining  Value  Tables 

When  a decision  is  affected  by  multiple  attributes 
(either  criteria  or  conditioning  elements) , VTs  must  be  com- 
bined to  arrive  at  a composite  expected  utility.  If  VTs  are 
constructed  separately  for  each  criterion  with  all  conditioning 
elements  being  represented  in  each  table,  then  the  general  form 
of  the  EU  calculation  can  be  described  as 


EU, 


ss 


w.  u. . 

J Ijm 


In  cases 
ditioning  elements 
instead  compute 


where  the  U. . are 
i^m 

jointly  (as  is  the 


not  assigned  to  all  con- 
case  in  Reference  1),  we 


EUi 


ss 


m=l 


+ 


+ 


The  principal  ground  rule  for  utilities  is  that  they 
must  represent  monotonic  measures  of  preferences  for  the  out- 
comes of  decision  actions  --  bigger  is  better.  Any  rules  for 
combining  must  be  consistent  with  this;  in  other  words,  it  must 
not  be  possible  after  combination  over  elements  of  one  case  to 
arrive  at  a utility  value  lower  than  that  for  a second  case  in 
which  the  preference  for  the  outcome  is  the  same  or  higher. 


2. 2.1.3  Time- Dependency 

Consider  some  simple  possibilities  where  the 
are  time -dependent.  Mission  A has  three  phases: 

(1)  Navigate  successfully  to  target. 

nn 
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(2) 

(3) 


Destroy  target. 

Return  safely. 

Let  all  elemental  utilities  be  in  the  range  0 (failure) 
to  1 (success).  As  a trial,  let  the  overall  utility  for  the  mis- 
sion be  the  weighted  sum  of  the  individual  utilities  Uj , U2,  Uj, 
with  weights  , W2,  (assigned  0.25,  0.5,  0.25).  For  the 
example,  allow  only  full  success  or  full  failure  as  illustrated 
below: 


UTILITY  ASSIGNMENTS 


POSSIBLE  OUTCOME  COMBINATIONS 


PHASE 

WEIGHT 

CASE  1 

CASE  2 

CASE  3 

CASE  4 

CASE  5 

1 

0.2s 

0 

1 

1 

1 

1 

(•nraut«| 

2 

O.S 

0 

0 

0 

1 

1 

(at  targat) 

3 

(raturn) 

Ovarall 

0.25 

0 

0 

1 

0 

1 

0 

0.26 

0.5 

0.76 

1 

•vfvhrti 

mhaloo 

JwttWwW, 

Mirrival 

mitiion 

MMOMS, 

no  survival 

mmion 

tUOOMS 

survival' 

A fully 

successful  mission  in 

all  phases  produces  a 

Utility 

equal  to  1 

, and  an 

unsuccessful 

mission 

yields : 

U - 

W2U2  - W3U3) 

- 0 

The  outcomes  of  Phases  2 and  3 are,  however,  dependent 
on  earlier  phases  so  that  not  all  combinations  are  possible. 


Successful  destruction  of  the  target  without  returning  safely 
yields  a utility  of  0.75.  But  higher  utility  is  assigned  to 
successful  navigation  to  target  without  target  destruction  and 
without  returning  safely  (0.25)  than  if  all  phases  were  unsuc- 
cessful (0).  While  this  is  arguable,  the  outcomes  are  effec- 
tively the  same  and 'should  receive  the  same  utility.  The  sig- 
nificance is  that  the  assignment  of  elemental  utilities  makes 
sense  only  if  such  assignments  are  associated  with  entities 
that  are  independent  in  terms  of  value  (rather  than  probability), 
such  as  enemy  destruction  and  survival.  Here,  weighted  sums  of 
utilities  have  useful  interpretations. 

Consider  another  method  for  combining  elemental  util- 
ities --  a product  rule.  We  can  define  U = XU2  xU^  and 
obtain  the  following  table  of  utility  assignments: 

UTILITY  ASSIGNMENTS 


POSSIBLE  OUTCOME  COMBINATIONS 

PHASE 

CASE  1 

CASE  2 

CASE  3 

CASE  4 

CASE  5 

1 Unroute) 

0 

1 

1 

1 

1 

2 (at  targat) 

0 

0 

0 

1 

1 

3 (ratiirn) 

0 

0 

1 

0 

1 

Oaarall 

0 

0 

0 

0 

1 

i 1 

t 
i 

I I If  all  U^'s  are  one,  U - 1 as  desired;  if  any  is 

I ‘ 0,  then  U - 0.  But  that  gives  the  same  utility  if  Phase  3 


f f fails  whether  or  not  the  target  is  destroyed.  It  also  gives 

^ I ' 

I I no  preference  to  survival  over  non- survival  if  the  target  is 

I not  ilAfitraved. 


Other  combinations  might  be  conceived,  such  as  a 
sum  whose  terms  contain  some  products  of  elemental  utilities, 
as  seen  in  the  following  example.  Let 

U = * W3U3 


and  assign 


Wj,2  = 0.7S 
W3  = 0.25 

Then 


UTILITY  ASSIGNMENTS 

POSSIBLE  OUTCOME  COMBINATIONS 


PHASE 

CASE  1 

CASE  2 

CASE  3 

CASE  4 

CASE  5 

1 Unroute) 

0 

1 

1 

1 

1 

2 (at  targat) 

0 

0 

0 

1 

1 

3 (rtturn) 

0 

0 

1 

0 

1 

Ovarall 

0 

0 

0.25 

0.75 

1.0 

mitfion  failura.  no 

Minrhral 

minion 

failura. 

wrvKral 

minion 
Micoan, 
no  survival 

minion 

Micoan, 

survival 

or,  with  Wj2  ” 0.5;  W3  = 0.5,  we  have 
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UTILITY  ASSIGNMENTS 

POSSIBLE  OUTCOME  COMBINATIONS 


PHASE 

CASE  1 

CASE  2 

CASE  3 

CASE  4 

CASE  5 

1 (•nroutt) 

0 

1 

1 

1 

1 

2 (at  target) 

0 

0 

0 

1 

1 

3 (raturn) 

0 

0 

1 

0 

1 

Overall 

0 

0 

0.S 

0.5 

1.0 

minion  failure,  no 
lurvival 

minion 

failure, 

survival 

mitsion 
tuccm. 
no  turvival 

rmuion 

Miccatte 

turvival 

Both  weighting  assignments  yield  a plausible  ordered 
utility.  However,  the  approach  which  guarantees  preserving  pre- 
ference ordering  is  one  of  assigning  utilities  to  the  allowable 
states  associated  with  the  various  applicable  conditioning  ele- 
ments without  segmenting  by  mission  phases. 

In  short,  it  may  be  possible  to  construct  consistent 
and  combinable  utility  functions  for  mission  phases,  but  the 
final  test  is  whether  the  overall  values  meet  the  conditions 
required  of  utilities.  The  procedure  of  separate  valuation 
is  not  recommended  as  a general  one. 

2.2.2  Requirements  for  a Generalized  Formulation 

The  preceding  discussion  shows  that  the  dependent 
properties  of  sequences  of  events  resulting  from  the  decision 
to  initiate  a time-extensive  action  place  certain  requirements 
on  a generalized  decision  procedure.  Additional  requirements 
are  imposed  if  the  pCSj^)  are  defined  by  conditioning  elements 
which  are  functions  of  time.  These  observations,  although 


related,  are  not  identical;  furthermore,  they  apply  to  any 
such  decision  whether  of  the  "best"  time  or  the  "best"  alter- 
native types,  as  described  in  Section  I. 


There  are  four  different  ways  that  time  can  enter 
decision  problems: 


(1)  Decisions  which  result  in  time- 
extensive  actions. 

(2)  Decisions  which  are  truly  the  same 
except  for  the  time  the  action  is 
effected. 

(3)  Decisions  as  above  except  the  action 
must  be  implemented  within  a certain 
range  of  times. 

(4)  Decisions  for  which  a choice  must  be 
made  within  a certain  time  irrespective 
of  when  the  action  is  taken. 


The  last  category  is  dismissed  immediately  since  it 
applies  to  the  speed  of  reaching  a selection,  which  can  be  seg- 
regated from  the  selection  optimization  in  a multi -decision 
structure. 


Decisions  which  result  in  time-extensive  actions  gen- 
erally depend  on  CEs  which  are  time-dependent  as  well.  This 
need  not  always  be  the  case;  however,  generality  demands  the 
capability  to  handle  such  cases.  Thus,  generally,  a CE  must 
be  defined  as  time-extensive.  Likewise,  definition  of  an  S 
must  include  specification  of  CEs  that  are  time-extensive. 

Decisions  which  are  truly  the  same  except  for  the 
time  the  action  is  implemented  also  require  consideration  of 
time-extensive  CEs.  If  the  decisions  are  truly  the  same,  then 
in  reality  only  a single  alternative  is  available.  If  time  is 
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isolated  as  a criterion,  then  other  criteria  can  be  restricted 
to  showing  time- independent  utilities.  Thus,  all  which  truly 
represent  a single  alternative  can  be  considered  as  a single 
for  all  criteria  except  timeliness. 

The  third  class  addresses  the  time  domain  of  interest 
within  which  a single  predetermined  alternative  must  be  selected 
to  optimize  EU  locally.  This  construct  requires  no  special 
treatment  other  than  to  recognize  that  not  all  A^(t)  may  be 
candidates  for  selection  due  to  external  constraints. 

To  summarize,  a generalized  model  capable  of  handling 
time-extensive/non-time-extensive  actions,  time-dependent/non- 
time-dependent parameters,  and  unlimited  numbers  of  criteria 
must  possess  the  following  properties: 

(1)  Utilities  rather  than  regrets  must  be 
used  as  the  value  measure. 

(2)  Values  must  be  assigned  to  each  alterna- 
tive for  each  criterion,  but  for  all  con- 
ditioning elements  specified. 

(3)  Probabilities  of  states  must  be  allowed 
to  be  time -dependent. 

(4)  The  interdependence  of  time-sequenced 
events  must  be  considered  in  computing 
P(S^). 

2.3  EXTENSION  TO  THE  GENERALIZED  OUTCOME  CALCULATOR 

Fortunately,  the  lessons  learned  in  examining  state- 
based  models  easily  extend  to  outcome  calculator  models. 

The  last  two  requirements  above  simply  imply  that  the  outcome 
calculator  must  have  the  capability  to  treat  dynamic  variables 
and  must  derive  inputs  to  time- sequenced  events  from  outputs 
of  precedent  events.  The  second  requirement  readily  translates 


1 
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to  assigning  values  to  outcomes  for  each  criterion.  The  first 
requirement  stands  unchanged. 

2.4  APPLICATION  OF  THE  AID  TO  OTHER  PROBLEMS 


2.4.1  The  State  Model 

The  state  model,  developed  f.or  the  decision  aid  des- 
cribed in  the  following  sections,  provides  a framework  capable 
of  handling  any  "best  time  to"  decision  which  involves  five  condi- 
tioning elements,  two  criteria,  six  possible  times,  and  twelve 
states  of  the  world.  By  changing  the  input  data  and  the  stored 
data  that  define  the  meaning  and  values  of  these  elements,  the 
decision  situation  being  addressed  can  be  readily  changed.  Only 
modest  programming  changes  would  be  required  to  vary  the  number 
of  conditioning  elements,  criteria,  and  states. 


The  use  of  breakpoints  as  model  inputs  in  defining  sub- 
jective terms  such  as  "good,"  "bad,"  "superior,"  etc.,  provides 
a ready  means  to  adapt  the  model  to  specific  aircraft,  crew, 
ordnance,  fatigue,  morale,  etc.,  capabilities.  Analogously, 
if  an  amphibious  assault  were  modeled,  similar  variabilities 
in  asset  performance  could  be  accommodated. 


Both  breakpoints  and  utilities  provide  a means  to 
tune  the  model  based  on  experience,  either  manually  or  auto- 
matically, by  use  of  an  adaptive  feedback  program  such  as  those 
considered  in  other  decision  aiding  studies. 

The  most  severe  limitation  in  generalizing  the  state 
model  results  from  maintaining  a tractable  number  of  states. 
This  observation  is  true  of  all  state  models. 
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Extension  of  the  model  to  multiple-choice  decisions, 
either  conventional  or  time-extensive,  can  be  achieved  by  pro- 
viding additional  VTs  and  iterating  the  final  expected  utility 
calculations  described  in  Section  2.2.  Significant  programming 
changes  would  be  required  in  input,  algorithm,  and  display  por- 
tions of  the  code. 

2.4.2  The  Outcome  Calculator 

Like  the  state  model,  the  outcome  calculator  provides 
a framework  capable  of  similar  quantitative  and  qualitative 
expansion.  The  outcome  calculator  has  been  implemented  as  a 
general  transfer  function  model  with  the  functions  represented 
by  stored  tables  of  values.  Modification  of  the  transfer  func- 
tions provide  the  same  flexibility  as  breakpoint  changes  in  the 
state  model,  although  their  complexity  makes  the  process  far 
more  time  and  thought  consuming. 

The  transfer  function  tables  could  easily  be  gener- 
ated by  static  or  dynamic  use  of  engagement  models  specifically 
suited  to  the  scenario,  force  mix,  environment,  and  other  fac- 
tors available  to  the  aid.  As  reported  in  "Augmentation  of  the 
Naval  Task  Force  Decision  Aiding  System:  The  Outcome  Calculator" 
(Reference  7),  dynamic  use  of  models  may  be  unrealistic  in  light 
of  the  inputs  present  models  require  and  the  cost,  in  time  and 
money,  of  using  them. 

Extension  of  the  outcome  model  to  multiple-choice 
decisions  is  far  easier  than  extension  of  the  state  model  since 
the  internal  logic  of  the  model  is  unaffected;  extension  is 
achieved  by  posing  additional  VTs  to  map  expected  outcomes 
into  composite  expected  utilities.  • 

i 

rsn  I 
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2.4.3 


Multiple-Choice  Extension 

To  draw  an  example  of  the  ability  to  extend  the  model 
to  mult iple- choice  problems,  let  us  consider  the  fairly  complex 
set  of  alternatives 

• Launch 

• Prepare  to  launch 

• Wait  and  examine  decision  later 

with  the  assumption  that  preparing  to  launch  requires  one  hour, 
and  launching  cannot  take  place  for  one  hour  after  deciding  to 
launch  a prepared  force  (therefore,  two  hours  later  for  an 
unprepared  force).  Let  us  consider  one  criterion,  namely  tar- 
get destruction,  applied  to  the  state  model.  For  simplicity, 
we  presume  the  states  of  the  world  are  defined  as 

• --  we  have  superior  air  power 

• --  We  have  equal  air  power 

• Sj  --  We  have  inferior  air  power 

with  air  power  being  the  only  parameter  of  consequence  in  our 
decision. 


We  now  ask  for  utility  assignments  (on  the  0 to  100 
scale)  for  each  combination  of  and  and  obtain,  for  example. 


Alternative 

Launch 

Prepare  to  Launch 
Wait 


State 

is 

100 

80 

40 

90 

70 

30 

60 

60 

60 
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Let  us  suppose  we  execute  the  algorithm  and  obtain 
the  P(Sjjj)  which  are  necessarily  t ime  - dependent  with  reference 
to  the  A^;  for  example,  the  P(Sj^)  for  are  for  time  now  plus 
1 hour,  for  A^  they  are  for  time  now  plus  2 hours,  and  for  A. 
they  are  for  time  now  plus  0 hours  (though,  for  A,,  the  utili- 
ties are  independent  of  the  state  that  occurs  and,  hence,  are 
unaffected  by  the  time  for  which  the  probabilities  are  computed). 

Applying  the  proper  P(S^)  to  the  using 

'\ 

EU.  = y*  p(S  ) u. 

1 ^ m^  im 

m 

i we  obtain  EU^,  EU,,  and  EUj.  This  allows  us  to  select  among 

i ' Aj^,  A,,  and  Aj  as  in  a conventional  decision  situation. 

( Furthermore,  we  can,  in  complete  analogy  to  the  pre- 


sent implementation,  provide  EU,  (t)  , EU,(t),  and  EU,(t),  allow- 
ing  selection  of  the  maximum  EU^Ctj)  --  the  best  action  and  the 
best  time. 


r 


III.  SELECTION  OF  GRAPHIC  DESIGN 


3.1 


NOMOGRAPHY 


Nomography  is  the  art  or  science  of  constructing 
graphs  that  enable  a user  to  determine  the  value  of  a dependent 
variable  from  the  value(s)  of  one  or  more  independent  variables 
by  use  of  an  ortholinear  projection.  For  the  purposes  of  this 
discussion,  a nomograph  will  be  understood  to  be  a CRT  graphical 
display  which  presents  to  the  decision  maker  'if)rnation  that 
relates  the  values  of  the  inputs  to  a decision  algorithm  to  the 
values  of  the  outputs  of  the  algorithm.  It,  thus,  allows  the 
decision  maker  to  examine  the  relationship  between  input  and 
output,  to  study  the  sensitivity  of  the  outputs  to  changes  in 
the  inputs,  and  to  conduct  "what  if"  exercises. 

A nomograph  is  generally  inapplicable  to  merely  dis- 
playing the  values  of  an  input,  since  no  rule  of  a predictive 
nature  is  implied.  Although  one  could  argue  that  causality 
and  trends  in  the  real  world  are,  in  fact,  the  "rules"  which 
produce  perceived  inputs  as  representations  of  underlying 
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elemental  processes,  this  is  nonetheless  totally  useless  to 
the  decision  maker.  His  concern  is  reaching  a decision 
for  which  the  outputs  of  the  decision  algorithm  provide  him 
some  aid.  If  the  outputs  can  be  provided  with  a rule  of  a 
predictive  nature,  then  a service  is  being  provided  to  the 
decision  maker. 

Thus,  the  term  nomography,  as  wo  will  use  it,  refers 
to  a method  of  displaying  the  results  of  an  algorithm,  where 
there  is  a predictive  rule  that  converts  input  data  to  output 
results . 


The  term  nomograph  is  usually  reserved  for  those 
graphical  presentations  in  which  the  values  of  two  or  more 
variables  are  required  to  select  the  resulting  dependent 
variable.  However,  it  is  not  necessary  for  all  independent 
variables  to  appear  physically  in  the  nomograph.  In  machine- 
supported  displays,  the  values  of  additional  variables,  though 
not  explicitly  shown,  can  be  used  to  determine  the  functional 
relationship  between  those  independent  variables  which  are 
input  to  the  nomograph  and  the  dependent  variable  which  is 
extracted  from  the  nomograph.  Classical  definitions  of  nomo- 
graphy would  typically  exclude  a two-axis  graph  in  which  an 
output  can  be  determined  by  specifying  one  input.  However, 
in  the  case  of  computer- supported  displays,  where  the  func- 
tional relationship  between  these  two  variables  is  dependent 
on  a number  of  independent  variables  which  have  been  set  pre- 
viously, the  resulting  graphical  presentation  is  nonetheless 
a nomograph. 

Thus,  two  of  the  displays  used  in  the  decision  aid 
can  be  appropriately  termed  nomographs:  the  sensitivity  analy- 
sis display,  an  example  of  which  is  presented  in  Plate  S and 
the  aircraft  lost  by  strike  display,  depicted  in  Plate  4.  The 
former  is  a one-quadrant  nomograph  entered  by  specifying 


-35- 


a single  variable  --  specifically,  the  selected  conditioning 
element  for  the  sensitivity  analysis.  The  aircraft  loss  nomo- 
graph uses  a multiple  quadrant  presentation  to  provide  own 
force  losses  as  a function  of  the  segments  of  an  air  strike 
for  a single  presumed  launch  time.  Both  analyses  assume  that 
all  other  variables  input  to  the  decision  algorithm  have  the 
t ime - dependent  distributions  specified  in  their  corresponding 
input  displays. 

The  sensitivity  display  provides  the  decision  maker 
with  the  ability  to  correlate  the  range  of  variability  of  the 
expected  outcome  utilities  with  the  domain  of  variability  of 
an  independent  variable.  It  allows  a decision  maker  to  examine 
how  dependent  his  expected  outcome  is  on  variations  in  that 
input  parameter.  For  those  inputs  which  represent  factors 
under  the  decision  maker’s  control,  such  as  number  of  aircraft 
launched,  the  sensitivity  analysis  provides  a means  to  deter- 
mine the  value  of  launching  more  or  less  than  the  desired  num- 
ber of  aircraft,  as  well  as  the  change  in  utility  that  would 
be  expected  should  the  number  of  aircraft  actually  launched  be 
fewer  than  the  number  predicted  to  be  available  for  launch  by 
the  maintenance  or  readiness  officer.  For  those  inputs  which 
represent  factors  beyond  the  control  of  the  decision  maker, 
such  as  enemy  ground  defenses,  the  decision  maker  is  provided 
with  a measure  of  the  impact  upon  expected  outcome  that  lack 
of  knowledge,  error  in  observation,  or  misjudgment  of  enemy 
readiness  will  have  upon  the  consequences  of  a strike  decision. 
Thus,  the  sensitivity  analysis  is  a predictive  tool  that  sup- 
ports the  decision  maker's  analysis  by  informing  him  of  the 
risks  inherent  with  each  available  alternative. 


The  loss  nomograph  provides  information  which  cannot 
be  provided  by  the  sensitivity  analyses.  Specifically,  it  indi- 
cates in  which  mission  segment  losses  are  occurring.  Therefore, 
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if  the  decision  maker  is  able  to  increase  his  strike  force, 
he  will  be  able  to  decide  whether  to  increase  air  superiority 
fighters,  EW  escorts,  bombers,  etc.  Further,  unlike  a gross 
loss  number,  the  segment -dependent  indication  of  loss  provides 
further  confirmation  to  the  decision  maker  that  the  expected 
outcomes  produced  by  the  model  are  realistic  with  respect  to 
each  segment  of  the  mission.  For  example,  in  an  evaluation 
in  which  the  decision  maker  has  placed  greater  weight  on  target 
destruction  than  on  own  force  losses,  he  may  be  concerned  about 
whether  target  destruction  is  achieved,  principally  in  the  air- 
to-air  combat  prior  to  reaching  the  target,  over  the  target  or 
during  the  return  leg,  or  by  bombing  aircraft  on  the  ground. 
Although  in  either  case  his  own  losses  may  be  equally  high, 
his  confidence  in  the  aid  may  be  reinforced  if  he  sees  that 
the  losses  occur  early  in  the  mission  rather  than  later  when 
high  losses  would  tend  to  reduce  his  combat  effectiveness. 

Of  course,  all  these  considerations  may  ascribe  too 
much  introspection  to  the  decision  maker  --he  may  neither  have 
the  time  nor  the  propensity  to  conduct  such  an  evaluation. 

This,  however,  is  a point  which  can  only  be  answered  experi- 
mentally. 

3.2  PHYSICAL  APPEARANCE 

All  of  the  graphical  displays  used  in  the  aid  have 
been  designed  to  take  advantage  of  the  RAMTEK  color  display 
capability  at  the  University  of  Pennsylvania  test  bed.  Color 
has  been  used  primarily  as  a discriminant,  to  aid  the  decision 
maker  in  spotting  the  important  information  in  the  display 
(see  Appendix  E) . Color  has  also  been  used  to  indicate  cri- 
ticality in  the  enemy  ground  defenses  display,  where  the  ratio 
of  red  to  blue  colors  serves  as  a quick  indicator  of  the  inten- 
sity of  expected  ground  defenses.  However,  except  for  those 
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displays  where  color  is  employed  as  a discriminant  to  compact  j 

i 

more  information  than  would  be  conveniently  displayed  in  a 
single  black  and  white  presentation,  all  displays  would  be  j 

equally  interpretable  in  black  and  white  based  on  simple  geo-  * 

metric  considerations. 

j 

Scaling,  letter  size  selection,  screen  size,  and 
other  symbology  have  been  selected  in  accordance  with  the 
capabilities  of  the  test  bed.  No  further  basis  is  claimed 

here  for  the  selections  made  in  the  design  of  the  graphical  I 

- 1 

displays  used  in  the  system. 

With  the  exception  of  the  loss  nomograph,  all  system  ^ 

displays  have  been  designed  with  the  independent  variables  on  ^ 

the  abscissa  of  the  graph  and  the  dependent  variables  on  the  -- 

ordinate.  This  complies  with  standard  mathematical  conventions 
and  the  customary  form  in  which  graphs  are  presented  in  Naval  ] 

publications  and  manuals.  Trial  implementations  in  which  the 
dependent  and  independent  variables  were  reversed  led  to  dis-  j 

orientation  or  confusion  on  the  part  of  observers,  confirming 
the  desire  to  maintain  a conventional  scale  orientation.  , 


The  loss  nomograph  provides  a display  of  the  cumula- 
tive effects  of  the  engagement  as  a function  of  time.  The 
display  has  been  designed  so  that  later  mission  segments  pro- 
gress in  a clockwise  direction.  Again,  this  selection  has 
been  made  in  accordance  with  normal  conventions  for  graphical 
presentation.  The  upper  left  quadrant  was  selected  as  the 
initial  point  for  entering  the  nomograph  because  the  eye, 
accustomed  to  reading  a page,  will  naturally  tend  to  fixate 
there  first.  One  could  also  argue  that  this  should  result  in 
a nomograph  that  is  best  entered  at  the  left  edge  of  the  upper 
right  quadrant;  this  hypothesis  can  be  evaluated  experimentally. 
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Orange  has  been  used  for  scale  indicators,  axes, 
and  labels  on  all  displays  because  it  draws  attention  and  at 
the  same  time  is  not  an  indicator  of  criticality.  In  those 
displays  in  which  only  one  additional  color  is  employed,  blue 
has  been  used  based  on  studies  of  color  radar  displays  that 
indicate  that  it  tends  to  minimize  eye  strain,  while  still 
attracting  attention,  and  being  readily  perceptible  against 
a black  background. 


Both  the  sensitivity  analysis  and  the  loss  by  segment 
nomographs  have  been  designed  so  that  projections  from  the  value 
of  an  independent  variable  to  the  value  of  the  dependent  variable 
are  consistently  performed  perpendicular  or  parallel  to  the  axes 
presented  in  the  graph.  Although  a large  number  of  nomographs 
in  the  literature  use  projections  oblique  to  an  axis,  an  exami- 
nation of  the  nomographs  in  several  Naval  operational  manuals 
has  indicated  that  parillel  or  perpendicular  projections  pre- 
dominate. Without  the  aid  of  additional  instruments  to  deter- 
mine the  correct  value  of  the  dependent  variable,  rectilinear 
projections  can  be  more  accurately  made  visually  than  oblique 
projections. 


3. 3 DISPLAY  OF  UNCERTAINTY 

As  was  noted  earlier,  uncertainty  as  used  in  this 
report,  represents  all  those  effects  that  lead  to  an  imprecise 
knowledge  of  the  real  world.  Thus,  uncertainty  can  result  from  i 

bias,  estimation  error,  statistical  variability,  imprecise  obser- 
vation and  confidence  (or  the  lack  thereof)  in  the  use  of  reported  j 

information.  While  knowledge  of  the  specific  reasons  for  the  ! 

uncertainty  or  its  distribution  are  not  intrinsic  to  the  aid  j 

itself,  performing  statistical  operations  on  inputs  with  uncer- 
tainties can  be  extremely  difficult  unless  some  a priori  assump- 
tions about  the  shape  of  the  distribution  functions  of  the  inputs 
are  made. 
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In  general,  the  decision  aid  treats  the  inputs  as 
expected  values  and  standard  deviations  of  normal  distributions. 
Based  on  this  assumption,  outputs  with  both  expected  values  and 
interquartile  ranges  can  be  presented. 

There  is  at  present  no  experimental  justification  for 
assuming  that  the  values  input  by  an  observer  would,  in  fact, 
be  the  mean  and  standard  deviation  of  a pre-known  distribution. 
Without  making  these  assumptions,  it  would  be  difficult  to  des- 
cribe statistically  what  has  been  calculated  as  a result  of  the 
mathematical  operations  performed  on  the  inputs  in  any  sense 
other  than  an  expected  value  and  its  perturbation.  Appendix  D 
presents  a more  detailed  discussion  of  this  problem.  Since  the 
problem  can  be  resolved  by  experimentation  and  its  results  incor- 
porated in  place  of  present  assumptions,  no  further  concern  is 
warranted  at  this  time. 
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IV.  USING  THE  DECISION  AID 


4.1  GENERAL  FEATURES* 

The  Analytics  Decision  Aid  (ADA)  has  been  designed 
to  minimize  the  difficulties  that  individuals  who  are  unfa- 
miliar with  computers  typically  experience  when  attempting 
to  run  a complex  program.  ADA  automatically  prompts  the  user 
for  the  appropriate  information,  supplies  help  if  the  user 
does  not  understand  available  options,  issues  error  messages 
when  the  user  enters  invalid  data,  and  warns  the  user  if  cal- 
culations are  being  performed  with  suspect  data.  The  user 
need  not  enter  all  data  every  time  the  aid  is  used  --  default 
data  is  automatically  supplied  or  the  user  can  save  the  data 
from  a previous  run  for  use  at  a later  time.  Each  of  the 
arrays  of  data  values  used  in  the  aid  are  individually  acces- 
sible and  modifiable,  obviating  the  need  to  enter  all  data 
items  in  order  to  modify  a single  table  or  even  a single 
number  within  a table. 

* ADA  is  currently  being  revised  and  expanded  to  support 
testing  now  underway  under  ONR  sponsorship. 


I 


4.2 


ACCESSING  THE  DECISION  AID 


ADA  resides  on  the  University  of  Pennsylvania's  DEC 
System  10,  in  two  versions.  One  version  uses  both  a standard 
alphanumeric  display  and  the  RAMTEK  color  graphic  display, 
the  other  uses  only  the  alphanumeric  display.  To  use  the 
RAMTEK  color  graphic  version,  the  user  must  be  at  the  Uni- 
versity of  Pennsylvania  and  connected  to  the  RAMTEK  color 
display  system.  The  alphanumeric  version  can  be  used  at  any 
location  having  access  to  the  DEC-10  via  a standard  teletype 
or  CRT  display. 


The  RAMTEK  version  is  run*  by  entering 
RUN  ADARAM  [4010 , 201  ] '*' 

after  signing  on  to  the  system.  The  tabular  version  is  run 
by  entering 

RUN  ADATAB  [4010,201] 

ADA  will  respond  with  the  message 


A 

i 

J 

1 

I 


ENTER  'READ'  TO  USE  DATA  FROM  LAST  RUN, 
'DEF'  TO  RUN  WITH  DEFAULT  DATA  - 


* In  the  instructions  that  follow,  a "carriage  return"  is 
implied  after  every  line  input  by  the  user. 

^ If  signed  on  under  account  4010,201,  the  entry  in  brackets 
is  not  necessary.  The  RAMTEK  must  also  have  been  previously 
assigned  to  the  job  by  entering 

ASSIGN  TTYXXX  8 


where  XXX  represents  the  unit  number  currently  assigned  to 
the  RAMTEK. 
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The  decision  maker  must  then  type  in  either  DEF  or 
READ.  If  DEF  is  entered,  the  default  data  set  (shown  in  Fig- 
ures 5 through  14)  will  be  used.  If  READ  is  entered,  the 
user  must  have  stored  data  during  a previous  ADA  run  on  a 
file  named  FOR20.DAT  (or  otherwise  identified  the  file  to 
the  system).  This  data  will  then  be  used  instead  of  the 
default  data.  ADA  will  tespond  with  the  message: 

ENTER  'HELP'  TO  OBTAIN  ASSISTANCE 
'RUN'  TO  BEGIN  EXECUTION- 

A novice  decision  maker  will,  at  this  point,  prob- 
ably enter  HELP,  even  though  entering  RUN  will  sequence  him 
through  operations  that  are  generally  self-explanatory.  The 
operations  that  result  from  typing  RUN  are  described  in  Section 

4.3. 


If  HELP  is  entered,  the  system  will  display  the 
table  shown  in  Figure  4.  This  table  lists  the  various  input 
and  output  tables  available  to  the  decision  maker.  The  table 
indicates  that  by  entering  the  appropriate  code  in  response 
to  the  SELECT  DISPLAY  cue,  the  decision  maker  can  examine  the 
inputs  to  the  aid,  run  the  models  to  obtain  the  decision  aid 
outputs,  and  run  sensitivity  analyses  to  examine  the  sensitivity 
of  the  results  to  changes  in  each  of  the  input  parameters. 

4. 3 THE  PREFERRED  STRIKE  TIME  DISPLAY 

The  PST  display  (see  Figure  5)  provides  information 
to  the  decision  maker  on  the  times  at  which  the  task  force 
commander's  subordinate  officers  last  provided  updates  to  the 
conditioning  elements  used  in  the  decision  aid  calculations 
and  enables  the  commander  to  request  that  this  information 
be  updated  before  the  decision  aid  is  run.  It  also  enables  , 

the  decision  maker  to  specify  the  earliest  and  latest  times  I 

nsn  i 
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BITEII  *HELP*  TO  OBTAIN  ASSISTANCE 
•RUN*  TO  BEGIN  EXECUTl  ON«HELP 

WHENEVER  THE  CUE  'SELECT  OISn.AY»*  APPEARS*  SELECT 
A DISPLAY  FRiH  THE  FOLLOWINO  LIST  BY  ENTERINQ  THE 

appropriate  abbreviation 

DISPLAY  abbreviation 


OWN  FORCE  READINESS 

OFR 

ENEMY  QROUND  DEFENSES 

BGD 

ENEMY  AIR  DEFENSES 

EAD 

weather  at  target 

WAT 

WEATHER  AT  CARRIER 

VAC 

STATE  MODEL  UTILITIES 

state 

OUTCOME  MOOB.  UTILITIES 
LIKELIHOOD  RATIOS  FOR  ENEMY 

OUTCOME 

GROUND  DEFENSES 

LIKELIHOOD  RATIOS  FOR  ENEMY 

LEOD 

AIR  DEFENSES 

LEAD 

PREFERRED  STRIKE  TIMES 

PST 

SENSITIVITY  analyses 

SENS 

TO  RUN  STATE  MODEL 

RUNS 

TO  RUN  OUTCOME  MODEL 

RUNO 

TO  CWPUTE  AIRCRAFT  LOSSES 

LOSS 

STOP  RUN 

STOP 

TO  REDISPLAY  THIS  TABLE 

HELP 

Figure  4.  The  HELP  Display  J 


Note:  Information  entered  by  the  decision  maker  is  underlined. 


SB.ECT  DISPLAY  ( OPR«  EQD*  BAO«VAT«tfAC#  STATS«  •UTCSME«LnD*LEAO« 
PST«SlllS#IUlS«RUNO«LtSS«STaP«  OR  HELP) -PST 


LAST  INTELLIQENCE  REPORT  AT 
ENEMY  AIR  DEPDISES  S6ft 

ENEMY  Q ROUND  DEFENSES  S6M 

LAST  VEATHER  REPORTS  AT 

CARRIER  SAOS 

TARSET  §6§9 

LAST  READINESS  REPORT  AT  0600 


DOES  THIS  INFORMATION  REQUIRE  UPDATlNO  BEFORE  THE  DECISION  AID  IS  RUNT- 

EARLIEST  POSSIBLE  STRIKE  TIME  0600 

LATEST  POSSIBLE  STRIKE  TIME  1100 

ALL  STRIKE  TIMES  ARE  ASSUMED  TO  BE  EQUALLY  ACCEPTABLE 

DO  YOU  WISH  TO  CHANGE  STRIKE  TIMESt  NO 

DO  YOU  WANT  TO  CHANGE  THE  PREFERENCE  FACTORS?  ^ 

ENTER  WEIGHTS  TO  BE  ASSIGNED  TO  EACH  ONE  HOUR  INTERVAL 

0600*^ 

0T00«*2 
06 00*77 
0000* l»0 
I 000*«8 
llOO*j|^ 

DO  YOU  WISH  TO  CHANGE  ANY  PREFERENCE  FACTORS?  NO 


Figure  5.  The  PST  Display 


NOTE:  Information  entered  by  the  decision  maker  is  underlined. 


at  which  the  strike  can  be  made  and  permits  the  assignment  of 
weights  to  each  strike  time  to  accommodate  his  personal  pre- 
ferences or  to  incorporate  dynamic  time-dependent  factors 
relevant  to  the  problem  that  have  not  been  explicitly  included 
in  the  decision  model. 


In  a typical  decision  process,  inputs  to  the  aid 
would  have  been  supplied  by  the  task  force  commander's  sub- 
ordinate officers,  e.g.,  the  intelligence  officer  would  have 
supplied  information  on  the  enemy  ground  and  air  defenses, 
the  weather  officer  would  have  supplied  weather  predictions 
at  the  target  and  at  the  carrier,  and  the  maintenance  or 
readiness  officer  would  have  supplied  own  force  readiness 
estimates.  The  task  force  commander's  primary  interest  is, 
therefore,  likely  to  be  in  the  time  at  which  these  data  were 
supplied,  relative  to  the  current  time.  This  summary  infor- 
mation is  available  in  the  preferred  strike  time  display. 

This  display  would  have  appeared  if  RUN  rather  than  HELP  had 
been  entered  when  the  decision  aid  was  originally  accessed. 

Initially,  the  information  shown  in  Figure  5,  the 

J 

times  of  the  last  intelligence,  readiness',  and  weather  reports, 
are  displayed  together  with  the  question 

DOES  THIS  INFORMATION  REQUIRE  UPDATING 
BEFORE  THE  DECISION  AID  IS  RUN?- 


If  the  decision  maker  feels  that  any  of  the  data 
requires  updating,  he  can  enter  YES.  The  system  will  then 
ask  him  to  enter  the  name(s)  of  the  conditioning  element (s) 
requiring  updates.  After  this  information  has  been  entered, 
or  if  no  updates  were  required,  the  system  will  display  the 
additional  information  shown  in  Figure  S,  along  with  the 
question 
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DO  YOU  WISH  TO  CHANGE  STRIKE  TIMES?- 

If  the  decision  maker  enters  YES  in  response  to  this 
question,  he  will  be  allowed  to  enter  new  earliest  and  latest 
strike  times.  Whether  or  not  he  chooses  to  change  the  strike 
times,  he  will  be  able  to  assign  weights  to  each  one-hour 
interval  or  to  weight  each  strike  time  equally  so  that  tac- 
tical objectives  which  are  time-critical  in  destroying  enemy 
assets  can  impact  evaluation  of  alternatives. 

Typically,  once  the  decision  maker  has  examined 
this  table,  he  will  proceed  to  run  either  the  state  model 
(by  entering  either  STATE  or  RUNS  in  response  to  the  SELECT 
DISPLAY  cue)  or  the  outcome  model  (by  entering  either  OUTCOME 
or  RUNO) . Entering  STATE  or  OUTCOME  ultimately  causes  the 
same  calculations  to  be  performed  as  RUNS  and  RUNO,  respec- 
tively. The  difference  between  the»«Dptions  is  that  STATE 
and  OUTCOME  provide  an  initial  display  of  the  state  and  out- 
come model  utilities  and  permit  the  decision  maker  to  modify 
these  utilities,  the  weights  assigned  to  the  two  criteria 
(target  destruction  and  own  force  loss)  and,  in  the  case  of 
the  STATE  model,  to  change  the  "breakpoints"  associated  with 
the  state  definitions.  Therefore,  in  the  following  discus- 
sion, the  STATE  and  OUTCOME  displays  will  be  described,  fol- 
lowed by  the  RUNS  and  RUNO  displays. 

4.4  THE  STATE  DISPLAY 

State  models  require  the  definition  of  a set  of 
states,  characterized  by  specific  values  for  each  of  the  con- 
ditioning elements.  The  ADA  state  model  currently  defines  12 
states  (Figure  6a),  based  upon  the  values  of  the  following 
conditioning  elements:  weather  at  target,  weather  at  carrier, 
enemy  ground  defenses,  and  relative  readiness.  (The  readiness 
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Figure  6(a).  The  STATE  Display 
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ratio  is  a combined  estimate  of  own  force  readiness  and  enemy 
force  readiness  [see  Appendix  B]).  Since  several  of  these 
quantities,  as  used  in  the  model,  are  continuous  variables, 
defining  12  states  requires  partitioning  the  ranges  of  the 
variables  into  discrete  values  and  forming  the  appropriate 
permutations  of  each  variable  with  every  other  variable. 

As  shown  in  Figure  6a,  ADA  partitions 


• The  continuous  variable  relative  readiness 
into  the  discrete  values  inferior,  equal, 
and  superior,  i.e..  Blue's  expected  force 
level  is  inferior,  equal,  or  superior, 
relative  to  the  expected  Orange  force  level. 

• The  continuous  variable  weather  at  target 
into  good  and  bad,  i.e.,  the  weather  at  the 
target  is  expected  to  be  either  good  or  bad 
with  respect  to  attaining  the  mission  objec- 
tives (destroying  the  enemy's  defensive/ 
offensive  capability). 

• The  continuous  variable  weather  at  carrier 
into  good  and  bad,  i.e.,  the  weather  at  the 
carrier  is  expected  to  be  either  good  or  bad 
with  respect  to  Blue's  ability  to  land  its 
planes  safely. 


The  values  of  the  conditioning  elements  that  partition 
the  ranges  are  termed  "breakpoints"  and  are  shown  in  Figure  6a. 

The  variable  enemy  ground  defenses,  as  defined  in  the 
model,  is  already  a discrete  variable  with  values  of  none,  low, 
medium,  or  high.  Thus,  as  indicated  in  Figure  6a,  state  12  can 
be  described  by  the  statements: 

• Our  air  power  is  superior  to  the  enemy's. 

• The  weather  at  the  target  is  good. 

• The  weather  at  the  carrier  is  good. 

• The  enemy  has  either  medium  or  high 
defenses. 


CK) 
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To  each  state  defined  in  this  way,  utilities  for 
target  destruction  and  own  force  losses  are  assigned  that  are 
interpreted  to  mean: 

"If  the  conditions  defining  this  state 
pertain  at  the  time  the  strike  occurs, 
the  target  destructioji  utility  that  will 

be  obtained  will  be  and  the  own 

force  loss  utility  that  will  be  obtained 
will  be 


Thus,  for  example,  if  the  real  world  is  in  state  12 
when  the  strike  takes  place,  a target  destruction  utility  of 
80  will  be  obtained  and  an  own  force  losses  utility  of  80  will 
be  obtained,  according  to  Figure  6a. 

The  state  definitions  must  be  mutually  exclusive  and 
exhaustive,*  and  the  values  of  utilities  assigned  to  the  states 
must  make  sense.  For  example,  it  is  clear  that  target  destruc- 
tion will  be  less  and  losses  are  likely  to  be  worse  (i.e., 
have  a lower  utility)  if  own  forces  are  inferior  rather  than 
equal  or  superior.  Therefore,  for  the  utility  assignments  to 
make  sense,  states  that  have  an  inferior  relative  readiness 
must  have  lower  target  destruction  and  own  force  loss  utili- 
ties. This  ordering  has  been  preserved  in,  for  example,  the 
utility  assignments  for  states  1 and  2 versus  states  3 and  4 
in  the  sample  set  of  utilities  shown  in  Figure  6a. 

The  state  model  calculates  the  probability  of  being 
in  each  of  the  12  states  at  each  of  the  specified  launch  times. 
From  these  probabilities,  it  determines  an  expected  utility 
for  launching  the  attack  at  each  time  by  multiplying  the 


1 i 

J 

I 

I I 


probabilities  by  the  associated  utilities  for  target  destruc- 
tion and  own  force  losses.  Before  performing  these  calcula- 
tions, however,  the  decision  maker  is  given  the  opportunity 
to  modify  the  utilities  associated  with  any  state,  the  break- 
points that  "discretize"  relative  readiness,  weather  at  target, 
and  weather  at  the  carrier,  and  the  weights  assigned  to  target 
destruction  and  own  force  losses. 

4.5  THE  OUTCOME  DISPLAY 

In  the  outcome  model,  utilities  are  assigned  to 
specific  target  destruction  and  own  force  loss  outcomes.  For 
example,  as  shown  in  Figure  7a,  if  an  engagement  with  the  enemy 
results  in  10  percent  of  the  target  being  destroyed  and  three 
of  our  own  aircraft  being  lost,  then  a target  destruction 
utility  of  50  and  an  own  force  loss  utility  of  65  will  be 
obtained.  The  outcome  model  (Appendix  C)  then  predicts  from 
these  utilities  and  the  probable  distributions  of  input  values, 
the  probable  distribution  of  outcomes  and  the  expected  utility 
distributions  for  striking  at  each  of  the  possible  strike  times. 
Prior  to  running  the  outcome  model,  the  decision  maker  is 
given  the  opportunity  to  modify  the  utilities  associated  with 
the  various  outcomes  and  the  weights  assigned  to  target  destruc- 
tion and  own  force  losses. 

4.6  EVALUATING  THE  DECISION  ALTERNATIVES  (RUNS  AND  RUNG) 

If  the  decision  maker  requests  either  the  state  or 
outcome  model  utility  displays,  ADA  allows  the  decision  maker 
to  modify  the  assigned  state  or  outcome  utilities  and  the 
weights  assigned  to  the  criteria  of  target  destruction  and 
own  force  losses.  It  will  then  automatically  calculate  the 
expected  utilities  and  standard  deviations  for  all  allowable 
strike  times.  Alternatively,  the  decision  maker  can  obtain 
the  state  or  outcome  model  expected  utilities  directly  (with- 
out reviewing  the  assigned  state  or  outcome  utilities)  by 
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Figure  7(a).  The  OUTCOME  Display 
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entering  RUNS  or  RUNG  when  the  SELECT  DISPLAY  cue  is  displayed. 

In  either  case,  the  tabular  display  generated  by  ADA  presents 
the  expected  utility  and  its  standard  deviation  for  each  time 
period  of  interest  (Figures  6b  and  7b).  The  corresponding 
graphical  presentations  are  shown  in  Plates  1 and  2. 

The  graphical  form  of  presentation  of  the  expected 
utilities  and  probable  ranges  for  each  of  the  decision  alter- 
natives enables  the  decision  maker  to  quickly  compare  the 
alternatives  and  then  select  the  one  he  considers  most  appro- 
priate. We  use  the  term  "appropriate"  here  because  although, 
in  the  long  run,  the  alternative  that  has  the  maximum  expected 
utility  is  theoretically  the  "best,"  it  may  be  the  case  that 
a decision  maker  will  not  select  this  alternative.  His  own 
attitudes  towards  risk-taking  or  his  knowledge  about  aspects 
of  the  situation  that  may  not  have  been  incorporated  in  the 
models,  may  cause  him  to  select  an  alternative  with,  e.g.,  a 
lower  expected  utility,  but  also  a smaller  probable  range. 

Such  a choice  is  not  necessarily  wrong;  in  fact,  depending 
on  the  circumstances,  it  might  be  a more  appropriate  decision 
than  selecting  the  alternative  with  the  highest  expected 
utility.  What  is  important  is  that  the  form  of  presentation 
used  in  the  aid  permits  the  decision  maker  to  evaluate  his 
alternatives  and  their  probable  consequences  with  respect  to 
a standardized  value  scale  that  clearly  tells  him  how  much  he 
can  expect  to  gain  or  lose  by  selecting  each  alternative. 

There  are  three  shortcomings  associated  with  the 
"Utility  vs.  Alternative"  presentations  shown  in  Plates  1 
and  2.  First,  the  decision  maker  has  no  way  of  determining 
where  the  utilities  are  coming  from  --  how  much  comes  from 
target  destruction  and  how  much  from  own  force  losses  -- 
because  these  figures  have  been  buried  in  the  overall  utility 
calculation.  Secondly,  the  utilities  that  are  displayed  are 
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meaningless  units  in  that  the  decision  maker  cannot  readily 
relate  them  to  the  quantities  "percent  target  destruction" 
and  "own  force  losses"  that  were  combined  to  arrive  at  the 
number.  Decision  makers  may  prefer  to  base  their  decisions 
on  numbers  that  they  can  understand,  rather  than  constructs. 
And  third,  many  decision  makers  want  to  know  what  accounts 
for  an  outcome,  how  sensitive  the  outcomes  are  to  errors  in 
estimation  of  the  inputs,  and  what  can  be  done  to  improve 
the  outcomes.  These  three  types  of  analyses  and  data  are 
supplied  to  the  decision  maker  in  the  sensitivity  analyses 
and  loss  nomograph  discussed  in  Sections  4.8  through  4.9. 

4.7  INPUT  DISPLAYS 

This  section  describes  the  displays  of  input 
quantities  available  to  the  decision  maker  and  his  subordinate 
officers.  The  expected  values  of  the  five  conditioning  ele- 
ments are  displayed  in  both  a tabular  and  a graphical  format. 
While  there  are  basic  similarities  in  each  of  the  formats, 
each  display  is  slightly  different,  depending  on  the  charac- 
teristics of  the  specific  variables.  Variables  with  differing 
characteristics  (discrete/continuous,  Bayesian  updated/non- 
updated,  etc.)  have  been  used  to  demonstrate  both  the  flexi- 
bility of  the  display  methodology  and  the  ability  of  the 
algorithms  to  handle  such  variations. 


4.7.1  The  OFR  Display  ! 

The  air  strike  mission  as  described  in  the  see-  , 

nario  specifies  a desired  mix  of  A-6,  A-7,  and  F-14  aircraft  ! 

to  be  used  to  carry  out  the  mission.  The  OFR  display  presents 
to  the  decision  maker  the  readiness  officer's  estimates  of 
how  the  probable  number  of  aircraft  available  at  each  of  the 
possible  strike  times  compares  with  this  desired  complement.  Q 

The  display  shown  in  Figure  8 indicates  the  time  of  the  report 

I 
I 


► 


s 

i 

t 


I 

1 

I 

I 

I 

I 

I 

[ 

[ 

i: 

r 

i: 

i; 

l: 

f: 

(i 

fi 

(i 


SELECT  DISPLAY  ( CFR,  EGD,  EAD,  WAT,  VAC/ STATE,  OUTCOME,  LEG  D,  LEAD, 
PST,  SENS,  RUNS,P.UMO,  LOSS,  STOP,  OP.  HELP)-OFP 

OWN  FORCE  READINESS  (OFP.) 

ne 

1 TIME  OF  REPORT  0600  NEXT  REPORT  AT  1233 

READY  AIRCRAFT  BY  TYPE 


TIME 


A- 6 A-7  F-IA 


2 

0600 

6+-  1 

12*-  6 

10+- 

3 

07  00 

7 + - 1 

1 A+-  a 

12+- 

4 

0800 

1 

164--  2 

1 A+- 

5 

0900 

9*-  1 

18  + - 2 

16+- 

6 

1000 

10+-  1 

20+ - 2 

16+- 

7 

1100 

1 1-t--  1 

22+-  2 

I6+- 

8 

AVAILABLE 

12 

2A 

IS 

9 

DESIRED 

12 

IS 

IS 

1 

2 

2 

2 

2 

2 


DO  YOU  WISH  TO  eiTE*»  A NE*!  PEPORT?  Xifl, 

DO  YOU  WISH  TO  CHANGE  A LINE  IN  THE  CURRENT  REPORT? 
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NOTE:  Information  entered  by  the  decision  maker  (or  a 
subordinate  officer)  is  underlined. 
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and  the  time  at  which  the  next  report  is  expected  (line  1). 
Then,  for  each  succeeding  one-hour  interval,  the  display 
indicates  the  expected  numbers  of  A-6,  A-7,  and  F-14  aircraft 
that  will  be  available  (lines  2 through  7).  Finally,  lines 
8 and  9 indicate  the  desired  strike  complement  and  the  total 
number  of  each  type  of  aircraft  available  in  the  task  force. 

The  corresponding  graphical  display  (Plate  11) 
presents  the  data  on  the  expected  availability  of  each  type 
of  aircraft  relative  to  the  number  of  aircraft  of  each  type 
desired  in  the  strike  complement.  The  allowable  strike  times 
are  indicated  along  the  x-axis.  The  available  aircraft  are 
shown  by  colored  vertical  bars,  each  aircraft  in  a different 
color.  The  horizontal  line  at  zero  on  the  y-axis  indicates 
the  desired  strike  complement.  Those  aircraft  distributions 
extending  above  the  line  indicate  times  at  which  the  number 
of  aircraft  of  that  type  may  exceed  the  number  desired  for 
the  mission. 

Those  distributions  extending  below  the  line 
indicate  times  at  which  the  number  of  available  aircraft  may 
fall  below  the  desired  strike  complement.  The  horizontal 
band  through  each  distribution  indicates  the  readiness 
officer's  best  estimate  of  the  number  expected  to  be  avail- 
able. This  number  is  interpreted  by  the  decision  algorithm 
to  be  the  mean  of  a normal  distribution  whose  a values  are 
shown  by  the  ends  of  the  bars. 

4.7.2  The  EAD  Display 

The  air  strike  mission  scenario  assumes  that 
Orange  will  keep  25  percent  of  its  available  planes  in  the 
air  at  all  times  and  that  Blue's  reconnaissance  flights  will 
report  back  every  four  hours  on  the  number  of  enemy  aircraft 
observed.  The  number  observed  therefore  serves  as  an  indicator 
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of  the  number  of  planes  that  Orange  has  available.  The 
decision  algorithm  uses  this  number  in  a Bayesian  updating 
process  to  update  a prior  estimate  of  the  enemy  aircraft 
distribution  to  provide  a posterior  estimate.  (The  likeli- 
hood ratios  used  to  perform  the  update  are  displayed  on  the 
LEAD  display  --  Figure  9b.)  The  posterior  estimate  is 
considered  to  obtain  for  the  four  hours  following  receipt 
of  the  report,  after  which  it  is  assumed  that  either  a 
new  report  will  have  been  received  or  the  enemy  will  revert 
to  his  standard  tactics.  Therefore,  for  launch  times  greater 
than  four  hours  after  the  most  recent  report,  the  prior 
distribution  is  used. 


The  EAD  display  (Figure  9a)  summarizes  these 
estimates  of  the  enemy  air  defenses: 


• Line  1 indicates  the  time  of  the 
report  and  the  time  at  which  the 
next  report  is  expected. 

• Line  2 indicates  the  number  of  air- 
craft reported  by  the  reconnaissance 
mission. 

• Lines  3 through  6 display  the  prior 
and  posterior  probabilities  that  the 
number  of  available  aircraft  are  within 
specified  values. 

• Line  7 indicates  the  useful  lifetime  of 
the  report. 

• Lines  8 and  9 indicate  the  mean  number 
of  enemy  aircraft  expected  to  be  avail- 
able for  the  four  hours  after  the  report 
and  the  number  expected  to  be  available 
for  times  greater  than  four  hours  after 
the  report. 


In  addition,  they  indicate  the  expected  numbers  of  aircraft 
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The  graphic  display  (Plate  12)  accompanying  this 
tabular  display  simply  shows  these  final  values  --  the 
expected  number  of  enemy  aircraft  and  the  25  percent  and  75, 
percent  cumulative  values  for  each  of  the  feasible  launch 
t imes . 

4.7.3  The  EGD  Display 

The  air  strike  mission  scenario  categorizes  the 
enemy  ground  defenses  expected  into  four  discrete  states: 
NONE,  LOW,  MEDIUM,  and  HIGH.  The  state  of  the  enemy's 
actual  defenses  are  indicated  by  the  intensity  of  defenses 
encountered  by  the  last  strike.  The  intensity  is  also 
categorized  into  four  states:  NONE,  LOW,  MEDIUM,  and  HIGH. 

A set  of  likelihood  ratios  (displayed  on  the  LEGD  display  -- 
Figure  10b)  relate  the  prior  estimates  for  the  enemy  ground 
defenses  to  the  posterior  estimates  that  would  obtain  after 
a Bayesian  update  based  upon  the  strike  debrief  estimates 
of  the  level  of  enemy  ground  defenses  encountered.  The  EGD 
tabular  display  (Figure  10a)  indicates  that  time  at  which 
the  last  report  was  entered,  the  time  the  next  report  is 
expected  (line  1),  the  estimate  of  enemy  ground  defenses 
based  on  the  strike  debrief  (line  2), and  the  prior  and 
posterior  probabilities  for  each  of  the  possible  states  of 
the  enemy  ground  defenses.  Since  it  is  assumed  that  the 
enemy  cannot  resupply  his  ground  defenses,  the  posterior 
probabilities  pertain  until  the  next  report  is  received. 

The  corresponding  graphical  display  (Plate  13) 
shows  the  cumulative  probabilities  of  each  of  the  states. 
This  display  enables  the  decision  maker  to  estimate  rapidly 
the  probability  of,  e.g.,  the  enemy's  defenses  being  no 
worse  than  LOW. 
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4.7.4  The  WAT  Display 

The  WAT  display  (Figure  11)  provides  the  decision 
maker  with  the  weather  officer's  estimates  of  the  prob- 
ability of  weather  suitable  for  the  air  strike  existing  at 
each  of  the  possible  arrive-at-target  times.  The  weather 
officer's  estimates,  in  that  they  are  probabilities,  are 
expressed  as  single  numbers.* 

The  associated  display  (Plate  14)  simply  plots 
these  values. 

4.7.5  The  WAC  Display 

Favorable  landing  conditions  depend  on  factors 
such  as  sea  state,  visibility,  etc.  Rather  than  modeling 
all  possible  factors,  the  aid  assumes  that  a landing  index 
consisting  of  a continuous  parameter  on  a scale  of  0 to  1 
could  be  constructed  from  these  factors  to  indicate  the 
probability  of  suitable  conditions.  In  addition,  it  is 
assumed  that,  because  of  the  vagaries  of  weather  and  the 
estimation  process,  one  could  assign  a range  of  probable 
error  to  this  estimate.  These  numbers  --  the  landing  weather 
index  and  its  probable  error  --  are  displayed  on  the  WAC 
tabular  display  (Figure  12)  and  the  corresponding  graphical 
display  (Plate  15) . 

4.8  SENSITIVITY  ANALYSES 

In  addition  to  the  expected  utilities  and  their 
associated  uncertainties,  the  decision  maker  is  likely  to  be 


Conceivably  the  officer  could  be  asked  to  express  them 
as  probabilities  and  probable  ranges  around  those  values, 
but  the  problem  that  would  result  from  compounding  a 
probability  distribution  with  another  probability  distri 


oution  would  be  severe. 
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Figure  11.  The  WAT  Display 


NOTE:  Information  entered  by  the  decision  maker  (or  a sub- 

ordinate officer)  is  underlined. 
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interested  in  the  sensitivity  of  the  results  of  the  decision 
aid  to  variations  in  the  input  quantities.  The  sensitivity 
analyses  available  in  the  decision  aid  give  him  the  capa- 
bility to  examine  the  effect  of  such  variations. 


There  are  seven  sensitivity  analyses  available: 
all  but  one  function  in  the  same  way.  Three  of  the  analyses 
examine  the  sensitivity  of  the  results  to  own  force  readiness 
by  varying  the  available  numbers  of  A-6,  A-7,  and  F-14  air- 
craft. Three  of  the  remaining  analyses  examine  sensitivity 
to  enemy  air  defenses,  weather  at  the  target  and  weather 
at  the  carrier.  Each  analysis  runs  the  decision  algorithm 
for  all  possible  values  of  the  parameter  under  study.  The 
final  analysis,  the  enemy  ground  defense  sensitivity  analysis, 
differs  from  the  others  in  that  it  requires  the  decision 
maker  to  enter  a hypothesized  ECU  distribution.  All  the 
analyses  assume  that  the  values  of  all  conditioning  elements 
other  than  the  one  under  study  are  to  be  held  constant  at 
the  values  that  they  have  at  the  launch  times  being  evaluated. 


The  sensitivity  analyses  are  invoked  by  typing 
SENS  in  response  to  the  SELECT  DISPLAY  cue.  The  decision 
aid  responds  with  the  message: 

SELECT  EITHER  A6,  A7,  F14,  EAD,  EGD,  WAT,  OR 
WAC  AS  THE  INDEPENDENT  CONDITIONING  ELEMENT 


If  the  decision  maker  selects  anything  except  EGD, 
the  aid  will  automatically  exercise  whichever  algorithm 
(either  the  STATE  or  OUTCOME  model)  was  run  last  for  all 
possible  values  of  the  independent  conditioning  element. 
Examples  of  the  resulting  tabular  displays  are  shown  in 
Figures  13a  through  13g.  The  corresponding  graphics  are 
shown  in  Plates  5 through  10.  Across  the  top  of  the 
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Figure  13(e).  The  EGD  Sensitivity  Analysis 
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Figure  13(f). 

The  WAT  Sensitivity  Analysis 
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Figure  13(g).  The  WAC  Sensitivity  Analysis 
NOTE:  Information  entered  by  the  decision  maker  is  underlined. 
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tabular  displays  all  the  allowable  strike  times  are  listed. 

Each  row  corresponds  to  a different  value  for  the  condition- 
ing element  under  consideration.  For  the  A-6,  A-7,  and  F-14 
analyses,  these  are  the  numbers  of  the  respective  aircraft 
actually  launched;  for  EAD,  each  line  is  an  assumed  number 
of  available  enemy  aircraft  (in  increments  of  10);  for  WAC 
and  WAT,  each  line  is  a successively  higher  value  of  the 
weather  index  (in  increments  of  0.1).  Entries  under  the 
column  headings  are  the  expected  utilities  and  their  stan- 
dard deviation. 

If  the  decision  maker  selects  the  EGD  sensitivity 
analysis,  the  aid  asks  the  decision  maker  to  enter  a hypothe- 
sized distribution  of  values  for  the  four  EGD  states.  It  then 
outputs  a table  similar  to  those  output  by  the  other  sensiti- 
vity analyses,  based  upon  the  hypothesized  distribution. 

4. 9 THE  LOSS  NOMOGRAPH 

When  the  outcome  calculator  is  used  as  the  under- 
lying decision  algorithm,  information  is  available  from  the 
aid  on  the  actual  expected  number  of  own  aircraft  lost,  seg- 
ment by  segment,  and  the  corresponding  expected  enemy  losses. 
The  former  information  is  available  to  the  decision  maker  in 
the  LOSS  nomograph;  the  latter  has  not  been  implemented  at 
the  present  time. 

The  LOSS  nomograph  (Plate  3)  is  a four-quadrant  nomo- 
graph, the  axes  of  which  represent  different  mission  segments. 
The  number  of  aircraft  launched  is  represented  along  the  x axis 
the  number  enroute,  along  the  *y  axis;  the  number  that  reach  the 
target, on  the  -^x  axis;  the  number  on  the  return  leg, on  the  -y 
axis;  and  the  number  that  land  safely,  on  the  -x  axis.  The 
decision  algorithm  plots  the  between -segment  losses  and  dis- 
plays them  by  a line  drawn  in  each  quadrant.  The  nomograph 
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is  used  by  hypothesizing  a number  of  aircraft  launched  and 
drawing  a line  from  the  corresponding  point  along  the  -x  axis 
parallel  to  the  y axis  until  it  intersects  the  line  in  that 
quadrant,  at  which  time  a line  is  drawn  parallel  to  the  x axis 
until  it  intersects  the  y axis.  The  process  is  repeated  around 
the  nomograph  until  the  -x  axis  is  again  reached.  The  resulting 
distance  between  the  "launched"  and  "landed"  values  on  the  -x 
axis  permits  a quick  estimation  of  the  number  of  Blue  aircraft 
lost. 

The  decision  aid  performs  this  process  for  the  deci- 
sion maker  automatically.  When  the  decision  maker  requests  the 
LOSS  nomograph,  the  aid  responds  with  the  instruction: 

ENTER  DESIRED  CALCULATION  TIME 


When  the  decision  maker  has  entered  a time,  the  aid 
draws  the  axes  and  the  reflection  lines.  It  then  asks  the 
decision  maker  to 

ENTER  NUMBER  OF  AIRCRAFT  LAUNCHED 


The  lines  corresponding  to  the  desired  number  of  air 
craft  are  then  drawn  for  the  decision  maker  (Plate  4).  Up  to 
six  values  can  be  displayed,  after  which  the  ai ^ automatically 
asks  the  decision  maker  to  select  a new  display.  The  number 
of  aircraft  expected  to  land  is  printed  on  the  tabular  display 
(Figure  14). 

4.10  SAMPLE  DECISION  PROBLEM 

As  an  example  of  the  evaluation  process  faced  by 
the  decision  maker  using  ADA,  consider  the  following  problem. 


! 


T.-r--'. 
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BEST  AVAIUBLE  COPY 


Sn.ECT  DISPLAY  ( tPR^ EDO# EAD* VAT^VAC* STATI« OUTCME.LnD«LCAD« 
PST#  SENS*  IVJIIS#  RUNE* LtSS«STfP«  M HELP>«»LtSS 

ENTER  DESIRED  CALCULATION  TINB-STfi 
ENTER  NUNBER  OF  AIRCRAFT  LAONCRiS^ 

EXPECTED  NUMBER  OF  AIRCRAFT  LANDED-  4 
ENTER  MUMKJt  OF  AIRCRAFT  LAUNCHED- 10 
EXPECTED  NUMBER  OF  AIRCRAFT  LANDED-  t 
ENTER  NUMBER  OF  AIRCRAFT  LAUNCHED- tS 
EXPECTED  NUMBER  OF  AIRCRAFT  LANDED-  IS 
ENTER  NUMBER  OF  AIRCRAFT  LAUNCHED-Sf 
expected  NUNBER  OF  AIRCRAFT  LANDED-  17 
ENTER  NUNBER  OF  AIRCRAFT  LAUMCHED-BS 
EXPECTED  NUMBER  OF  AIRCRAFT  LMIOED-  SO 
ENTER  NUMBER  OF  AIRCRAFT  LAUNCHED- SO 
EXPECTED  NUMBER  OF  AIRCRAFT  LAMOBO=~  SS 


Figure  14.  The  LOSS  Display 


NOTE:  Information  entered  by  the  decision  maker  is  underlined. 
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The  decision  maker  (the  task  force  commander)  has  just  been 
informed  by  his  senior  operations  officer  that  the  assault 
force  which  had  debarked  earlier  in  the  day  arrived  on  land 
and  was  progressing  toward  their  objective.  In  order  for 
the  ground  assault  force  to  reach  their  objective  by  1400 
hours  the  following  day,  they  would  cross  the  main  bridge 
early  in  the  morning.  The  intelligence  officer  reported 
that  an  enemy  mechanized  division  appeared  to  be  aware  of 
the  assault  force  objective  and  was  progressing  toward  the 
same  bridge  with  an  ETA  of  1100  hours.  The  operations 
officer  properly  concluded  that  an  air  strike  to  destroy 
the  bridge  after  friendly  forces  had  crossed,  but  prior  to 
the  enemy  mechanized  division's  crossing  was  desirable  and 
brought  the  problem  of  ordering  an  air  strike  to  the  task 
force  commander's  attention. 

The  task  force  commander  was  aware  that  enemy 
forces  maintained  a continuous  air  intercept  force  around 
the  bridge  area.  A successful  bombing  run  would  require  a 
complement  of  18  air  superiority  fighters  and  12  or  more 
fighter  bombers.  Because  of  the  uncertainty  of  crossing 
time  both  for  friendly  and  hostile  forces,  the  task  force 
commander  has  a great  preference  to  conduct  the  air  strike 
at  0900;  a lesser  but  significant  preference  for  conducting 
it  at  0800  or  1000;  and  feels  that  conducting  the  strike 
at  0700  or  1100  is  of  marginal  positive  value.  Due  to  the 
limitations  of  the  crossing  times,  there  are  no  other 
practical  times  worthy  of  consideration. 


The  task  force  commander  calls  for  execution  of 
ADA  with  time  preference  factors  set  to  the  following  values 
for  the  period  0600  through  1100  hours  --  0,  0.2,  0.8,  1.0, 
0.8,  0.2.  He  begins  execution  of  the  aid  and  asks  his  intel- 
ligence officers  to  provide  their  most  current  inputs  on 
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available  ovm  forces,  enemy  airborne  fighter  capability  and 
weather,  both  at  the  target  and  landing  areas.  After  examin- 
ing the  value  assignments,  the  commander  expresses  satisfac- 
tion with  present  values  and  weights  and  commences  execution 
of  ADA. 


ADA  presents  its  principal  output  to  the  commander. 
The  expected  utilities  and  their  range  of  variation  pre- 
dicted by  the  aid  are  virtually  identical  for  the  strike 
times  0800,  0900  and  1000.  However,  upon  examining  the 
possible  variation  of  the  results  at  1000,  the  commander 
estimates  this  as  a less  desirable  option.  He  is  thus  faced 
with  the  choice  of  selecting  0800  or  0900. 


In  order  to  make  this  selection,  the  commander 
calls  for  a display  of  the  sensitivity  of  the  results  to 
knowledge  of  the  enemy's  airborne  fighter  force.  The  commander 
knows  that  this  input  is  based  on  aging  intelligence  data 
which  he  considers  highly  suspect.  The  sensor  system  used 
to  obtain  this  data  has  recently  given  erratic  responses. 

The  commander  observes  that  the  utilities  predicted  by  ADA 
correspond  to  a point  in  the  sensitivity  analysis  for  0800 
where  a variation  of  only  20  percent  in  the  intelligence 
estimate  would  significantly  alter  expected  results.  However, 
a similar  impact  is  not  observed  at  0900.  This  leads  the 
commander  to  request  display  of  the  enemy  air  defense  data 
input  to  ADA.  Upon  seeing  this  display,  the  commander 
observes  that  the  intelligence  officer  has  entered  a con- 
stant expected  value  with  an  increasing  variation  with  time. 
However,  the  variation  assigned  for  0800  hours  is  deemed 
overly  optimistic  given  the  recent  reliability  of  the  sensor 
system.  The  commander  therefore  changes  the  input  assump- 
tion to  allow  for  a larger  possible  variation  and  re-executes 
the  aid.  The  primary  output  display  now  clearly  predicts 


i 

i 


j 


B 

B 

[I 


-72- 


or  variation  win  occur  at  uyuu. 


The  commander  now  wishes  to  examine  whether  any 
increase  in  his  planned  strike  force  will  give  him  a further 
guarantee  of  achieving  his  objective.  He  therefore  calls 
for  the  loss  nomograph.  The  loss  nomograph  shows  him  that 
I of  the  losses  he  can  expect  to  suffer  in  executing  the  air 

strike,  the  major  component  occurs  inbound  toward  the  target 
rather  than  over  the  target  area.  The  task  force  commander 
therefore  concludes  that  by  increasing  the  number  of  fighter 
j bombers,  he  can  saturate  the  inbound  airborne  defenses  and 

' allow  a larger  number  of  bombers  to  reach  the  bridge  area 

^ unimpeded  by  further  enemy  defenses.  Accordingly,  the  task 

( force  commander  decides  to  launch  his  air  strike  such  that 

the  bridige  will  be  destroyed  at  0900  hours  and  to  increase 
I his  planned  force  mix  to  use  all  available  fighter  bombers. 

I This  discussion  is  a hypothetical  example  of  how 

the  decision  aid  could  be  used  in  support  of  a tactical 
( decision  problem.  It  is  quite  similar  to  the  prototype 

^ implementation  of  ADA,  but  is  enough  removed  to  emphasize 

/ the  tactical  bases  and  methods  of  use  of  many  of  the  features 

( that  have  been  incorporated  in  the  aid. 

1 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


5.1  CONCLUSIONS 


In  developing  the  nomograph  decision  aid  des- 
cribed in  this  report  five  major  goals  were  accomplished. 


• Development  of  a decision  aid  that 
permits  the  generalization  of  the 
parameters  in  the  decision  algorithm, 
e.g.,  unlimited  numbers  of  condition- 
ing elements,  use  of  discrete  and 
continuous,  Bayesian  and  non-Bayesian 
updated  parameters,  etc. 

• Incorporation  of  uncertainty  in  inputs 
in  such  a way  that  it  is  clearly  repre- 
sented in  the  outputs. 

• Inclusion  of  a capability  for  han- 
dling problems  in  which  the  states  of 
the  conditioning  elements  are  time- 
dependent  . 

• Development  of  display  techniques 
that  are  readily  understandable  and 
interpretable  by  the  target  population 
of  decision  makers. 
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II 


• Development  of  a methodology  that 
can  be  adapted  to  different  decision 
making  algorithms,  e.g.,  both  state 
models  and  outcome  calculators,  with 
inputs  and  outputs  that  can  be  readily 
related  to  real-world  quantities. 


During  the  development  of  the  decision  aid,  we 
had  the  opportunity  to  observe  the  behavior  of  several  dif- 
ferent decision  makers  as  they  became  acquainted  with  the 
aid.  These  experiences  led  to  the  following  conclusions 
which  are  probably  useful  considerations  with  respect  to 
any  decision  aid  being  developed: 


• Presentation  of  the  potential  varia- 
bility of  an  outcome  gives  a decision 
maker  the  opportunity  to  take  risks 
from  a more  informed  perspective. 

• The  ability  to  examine  the  sensitivity 
of  an  outcome  to  changes  in  inputs 
enables  the  decision  maker  to  consider 
alternative  ways  to  accomplish  his 
goals  by  redirecting  his  resources;  it 
also  enables  him  to  develop  a better 
feel  for  the  possible  perturbations 

in  outcomes  that  could  be  expected  due 
to  changes  in  the  input  conditioning 
elements. 

• The  ability  to  describe  the  outcomes 
in  concrete  terms,  e.g.,  "own  aircraft 
lost"  and  "percent  target  destruction," 
add  to  the  acceptability  of  the  aid. 

• It  is  very  difficult  for  a decision 
maker  to  establish  and  use  a con- 
sistent set  of  rules  governing  the 
association  of  utilities  with  states; 
assignments  of  utilities  to  outcomes 
are  much  more  readily  and  consistently 
achievable  (and  are  more  clearly 
interpreted  as  a common  unit  for  multi- 
attribute combination  as  would  be 
equivalent  aircraft  or  equivalent 
dollars  for  example). 
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5.2  RECOMMENDATIONS 


There  are  two  major  courses  of  action  to  pursue  -- 
experimental  evaluation  and  development/refinement.  Since 
further  development/refinement  should  logically  take  advan- 
tage of  lessons  learned  in  experimental  evaluation,  Analytics 
recommends  that  an  experimental  evaluation  program  be  under- 
taken to  determine: 


• How  specific  design  variations  in 
graphic  presentation  (such  as  use 
of  color,  complexity  of  displays, 
output  of  physical  attributes 
versus  or  in  conjunction  with 
utilities)  impact  performance. 

• Whether  flexibility  as  embodied  in 
the  aid  is  an  asset  to  decision 
makers. 

• How  the  state  model  compares  with 
the  outcome  model. 

• How  the  nomograph  display  technique 
compares  with  other  approaches  such 
as  those  used  by  DDI  (Reference  1) 
and/or  NPRDC  (Reference  2). 


Further,  during  such  evaluation  preliminary  informa- 
tion on  cost/benefit  should  be  accumulated  to  aid  ONR  in  an 
overall  evaluation  of  alternatives  for  automated  decision 
aiding. 
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APPENDIX  A.  THE  SCENARIO 


A. 1 SOURCE  MATERIAL 

The  scenario  employed  in  developing  the  specific 
problem  implemented  in  the  Analytics  decision  aid  was  derived 
from  Stanford  Research  Institute  Memorandum  NWRC-RM-83,  ONRODA 
Warfare  Scenario  (Reference  2).  The  portion  of  the  scenario 
that  concentrates  on  air  strikes  on  the  ONRODA  airport  begins 
on  page  55  of  the  SRI  publication.  The  air  strikes  are  addressed 
as  Phase  I of  operations  in  OPORDER  CTF-1,  No.  1-7X,  and  its 
annexes  (especially  Annex  E) . The  scenario  is  briefly  summa- 
rized below. 


A.  2 THE  SETTING 

CTF-1  is  situated  400  nmi  west  of  ONRODA  Island  as 
shown  in  Figure  1.  CTF-1  consists  of  two  CVs  each  with  twelve 
A-6,  twenty-four  A-7,  and  eighteen  F-14  aircraft;  there  are 
additional  support  aircraft.  Both  CVs  have  off-loaded  RA-5C 
aircraft  at  Mid-Ocean  Island  to  assist  in  scheduled  reconnais- 
sance operations. 
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Daylight  extends  from  0600  to  1800  hours  daily. 
Weather  in  the  area  is  generally  clear  with  winds  variable 
from  12  to  15  knots  NW.  There  are  occasional  frontal  passes 
with  associated  light  rain  and  20-  to  25-knot  SW  winds. 
Fronts  generally  pass  in  12  hours  or  less.  During  frontal 
passes,  there  are  brgken  clouds  above  4000  to  5000  feet. 

On  approximately  10  percent  of  all  mornings,  there  is  ground 
fog. 


The  ONRODA  terrain  is  quite  flat.  The  airport  is 
located  in  rolling  hills  no  higher  than  250  feet  above  sea 
level . 


Intelligence  estimates  indicate  an  enemy  fighter 
capability  of  120  planes  located  on  the  island.  There  are 
also  37  SAM  and  AA  batteries.  The  Orange  air  force  conducts 
routine  strikes  on  Gray  cities  from  the  ONRODA  base  of  oper- 
ations . 


A regular  schedule  of  RA-5C  flights  has  been  estab- 
lished to  give  updated  estimates  of  Orange  air  activity  and 
weather  at  ONRODA  every  4 hours. 

A. 3 CONCEPT  OF  OPERATIONS 

Blue  air  forces  will  be  launched  in  Alpha  strike 
complements  of  six  A-6,  eighteen  A-7,  and  eight  F-14  aircraft 
by  each  CV  including  all  supporting  air  platforms.  Due  to 
political  considerations,  all  flights  are  restricted  to  oper- 
ate north  of  latitude  NS.  The  oporder  calls  for  ground  level 
approach  from  outside  radar  detection  range  as  well  as  avoid- 
ance of  Red  picket  ships  outside  the  strike  perimeter. 

The  strike  objectives  are  destruction  of  aircraft 
and  ground  defenses.  Secondary  objectives  will  be  added  in 
later  days  of  the  strike. 
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The  oporder  calls  for  2 Alpha  strikes  per  carrier  | 

per  day  during  daylight.  Aircraft  ordnance  will  consist  pri- 
marily of  LGBs  and  visual  acquisition  weapons.  Since  strike  .• 

mission  time  is  2h  hours,  a minimum  strike  cycle  time  of  5 
hours  is  recommended.  The  maximum  number  of  aircraft  consis- 
tent with  the  desired  complement  must  be  launched.  ! 


On  the  first  day  of  the  campaign,  each  CV  launches 
2 Alpha  strikes  at  predetermined  times  to  achieve  surprise. 
On  subsequent  days,  only  1 CV  will  launch  strikes  while  the 
other  performs  the  air  defense  mission.  The  CVs  will  alter- 
nate functions. 


Pilot  debrief  will  be  available  no  less  frequently 
than  every  18  hours  on  AA  counterfire  experienced  in  the  prior 
raid.  By  direction,  the  readiness  officer  will  provide  updates 
at  least  at  0600,  1200,  and  1800  hours  including  the  period 
through  the  next  report. 

Due  to  concurrent  blockade  operations,  it  is  presumed 
the  enemy  cannot  resupply  lost  aircraft  or  defenses. 


] 
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APPENDIX  B.  STATE  MODEL  ALGORITHM 


B.l  INTRODUCTION 

This  appendix  presents  a single  thread  description  of 
the  state  model  algorithm  used  in  ADA,  which  is  based  upon  a 
model  suggested  by  DDI  (Reference  1)  and  extended  by  Analytics 
to  include  both  larger  numbers  of  conditioning  elements  and  con- 
tinuous measures  of  the  state  of  the  world.  This  model  includes 
the  following  features: 


• Unlimited  number  of  alternatives. 

• Unlimited  number  of  conditioning  elements. 

• Continuous  conditioning  elements  as  well 
as  discrete  many-state  elements. 

• Time- dependent  states  of  the  world. 

• Time-extensive  alternatives. 

• Unlimited  number  of  criteria. 


nn 
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• Time  preference  for  action. 

• Bayesian  probability  updating. 


Because  of  constraints  imposed  by  the  test  implemen- 
tation, specific  limits  were  set  on  those  parameters  with 
potentially  unlimited  numbers.  The  technique,  however,  does 
not  have  any  of  the  limitations  implied  by  these  selections. 

The  reader  is  referred  to  the  DDI  report  cited  in 
Reference  1 for  the  development  of  the  methodology  used  here. 
That  reference  also  explains  the  application  of  multi-attribute 
utility  assessments  and  Bayesian  updating  as  incorporated  here. 

The  scenario  is  described  in  Appendix  A. 


CRITERIA 


Elemental  utilities  will  be  assigned  for  two  cri- 


teria (C)  : 


• Target  destruction  . 

• Loss  of  own  force  resources  (Cj^) 


Weights  W.J.JJ  and  Wj^,  subject  to  the  constraint  W.j,p  + = 1 , 

will  be  used  to  combine  the  utilities  associated  with  each 
criterion  into  a composite  expected  utility.  Default  values 
for  these  weights  will  be  3/4  and  1/4,  respectively. 


B. 3 MISSION  PROFILE 

The  mission  consists  of  five  segments  whose  times 
relative  to  the  time  of  launch,  L,  are  listed  in  Table  B-1. 
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B. 4 CONDITIONING  ELEMENTS 

Tiie  conditioning  elements  used  in  the  algorithm 
are  shown  in  Table  B-2. 

B.4.1  Readiness 

B.4.1.1  OFR 

It  is  presumed  that  one  of  the  CVs  in  CTF-1  will 
provide  the  support  aircraft  needed  for  the  strike  (one  E-2, 
one  EA-6,  and  three  KA-6) . 

For  each  time  interval  (one  hour)  in  the  planning  j 

range  (six  hours) , the  readiness  officer  will  be  requested  to 
input  the  expected  value  and  probable  error*  of  aircraft  avail-  \ 

able  by  type  in  response  to  questions  of  the  form: 

At  0600  hours,  what  will  the  expected 

number'  of  A-6s  available  be?  

with  probable  error  ± . 

The  readiness  officers'  responses  are  used  to  gen 
erate  a table  of  the  form  shown  in  Figure  B-1.^ 

These  values  are  considered  to  be  the  mean  (p)  and  | 

standard  deviation  (o)  of  a discrete  distribution  (whose 
envelope  is  the  normal  distribution)  truncated  over  the  ranges  J 

i| 
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I TABLE  B-2.  CONDITIONING  ELEMENTS  USED  IN 

DECISION  ANALYTIC  ALGORITHM 


SEGMENT 

NAME 

DESCRIPTION 

RANGE 

DISTRIBUTION 

BAYESIAN 

' 

Fighter  Own 

Force  Reedinett 
(OFRpI 

Number  of 

F-14 

0-18 

Normal  in,  a) 
Discrete 

No 

Bomber  Own 

Force  Reedineu 
(OFRg) 

Number  of  A-7 
Plus  A-6 

0-36 

Normal  {(i,  o) 
Discrete 

No 

II 

Enemy  Force 
Reedinett  — 

Enroute  (ERF||) 

1/16  of  avail- 
able fighter 
aircreft  on 
ONRODA 

0-5 

Normel  (/i,  o) 
Discrete 

Yes 

III 

EfMmy  Force 
Reedinen  ct 

Terget  (EFR|||) 

3/16  of  avail- 
able fighter 
aircraft  on 
ONRODA 

0-15 

Normal  (/i.  a) 

Yes 

Enemy  Ground 
Oefemet  (EGOI 

Ground  AA 

Four  States 

None 

Low 

Moderate 

High 

Yes 

Weetber  at 

Target  (WAT) 

Targat  visibility 
index 

0-1 

Probabilistic 

No 

IV 

Enemy  Force 
Readinen  Return 
(EFR,y) 

0.194  of  avail- 
able fighter  air- 
craft 

0-16 

Normal  (ju,  a) 
DiMrata 

Y« 

V 

Weather  at 

Carrier  (WAC) 

llaiMling  weather 
index 

0-1 

Normal  (/i.  a) 
Continuous 

No 

• 1 


• 1 


w 


1 

I 


I 

! 

I 


I 

! 

( 

I 

1 


,r 


i 


I 

I 

(i 

0 

0 


0 to  12  (A-6),  0 to  24  (A-7) , and  0 to  18  (F-14).*  The  fighter 
own  force  readiness  is  simply  the  number  of  F-14s; 

OFRp  = 0FRp_2^^ 

The  A-6  and  A-7  are  combined  to  form  a bomber  own 
force  readiness  estimate: 


OFRg  = OFR^.^  - OFR^.7 


Thus,  from  Figure  B-1  and  the  fact  that  if  A and  B are  inde- 
pendent anj 


C = A + B 


then 


0 


C 


and 


= ^A  ^ ^^B 

a combined  OFR  table  of  the  form  shown  in  Figure  B-2  can  be 
fo rmed. 


• Truncation  is  performed  over  the  range  of  available  air- 
craft on  the  CV. 

fTTl 
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B.4. 1.  2 


Relationship  Between  HAD,  EFRjj,  EFRjjj,  and  EFRjy 

Based  on  the  scenario,  it  is  assumed  that  only  air- 
craft will  be  encountered  enroute.  After  S-day,  the  enemy 
has  between  0 and  80  fighters.  Given  a y and  a,  and  assuming 
a normal  distribution,  a prior  P for  n = 0,  1,  ...,  79,  80 
can  be  generated.  The  default  distribution  used  in  the  deci- 
sion aid  has  a y of  50  and  a a of 

Intelligence  on  enemy  aircraft  comes  from  RA-5C 
reconnaissance.  They  report  every  4 hours  on  target  condi- 
tions, including  fighters  airborne  for  intercept.  Since 
Orange  can  expect  attack  at  any  time  of  day,  it  is  assumed 
that  he  will  keep  1/4  of  his  available  fighters  airborne. 

It  is  also  assumed  that  the  fighters  are  distributed  such 
that  1/4  of  those  airborne  (1/16  of  those  available)  will 
be  encountered  enroute,  thus 

EFRii  = ^ EAD 

If  the  states  of  the  relevant  indicator  for  EAD 
(up  aircraft  seen  by  RA-5C)  are  ^ 5 , 6 to  10,  11  to  15,  and 
^ 16,  the  general  appearance  of  the  EAD  Bayesian  updating 
process  would  appear  as  in  Table  B-3. 

It  is  assumed  that  the  intelligence  data  has  a 
useful  lifetime  of  only  4 hours.  Therefore,  when  a decision 
is  to  be  made,  for  all  launch  times  ^ 170  minutes  after  the 
time  at  which  the  reported  indicator  was  obtained,  the  pos- 
terior probabilities  obtained  from  applying  the  likelihood 
ratios  to  the  priors  are  used.  For  all  times  > 170  minutes, 
the  prior  probabilities  are  used.  (The  170-minute  breakpoint 
derives  from  the  4-hour  useful  lifetime  minus  the  70  minutes 
required  for  the  completion  of  Segment  II  after  the  start  o 
the  mission. ) 
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